Last month we presented a real CLEC’s unpleasant experience cutting over their PSTN circuit customers to VoIP (see Measuring VoIP Voice Quality ). Their main problem? How to measure and fine-tune VoIP call quality. To set the stage for this and remaining columns in this series, we divide the evaluation of voice quality in a VoIP network into two distinct tasks: (1) measuring the quality of the network segments, and (2) measuring the quality of the endpoint (IP phones) or gateway devices. If either area falls short, there’s little you can do with the other to compensate. First, let’s review some speech quality testing basics.
Quality Measurement Techniques
The ultimate measure of speech quality must relate back to the quality perception of live listeners. While automated quality evaluation techniques have become popular, the ultimate standard for subjective speech quality assessment of telecom systems is the Mean Opinion Score test, or MOS. As defined in the ITU recommendation P.800, an MOS test compares transmitted reference speech samples with the received speech and produces a numeric score from 1 (poor) to 5 (excellent) that relates to the perceived listener quality. Because MOS testing requires carefully prepared and controlled test conditions, the best way to get an MOS test done is to have it outsourced. Dynastat (Austin, TX – 512-476-4797, www.dynastat.com) is one company that has been in the speech evaluation business for years and is highly qualified to do MOS testing.
When live listener testing costs or time constraints are an issue, objective (automated, numerical comparison) techniques can be used to estimate the perceived quality of speech. Among these, PESQ (Perceptual Evaluation of Speech Quality, as defined in ITU-T P.862) and PAMS (Perceptual Analysis Measurement System) are the most popular in use today. Both involve playing reference speech or special audio prompts through the telecom system under test and comparing it with the “processed” audio collected at the receiving end.
Testing network by yourself
As a provider of VoIP telephone services, you could conduct a relatively simple automated end-to-end quality test on your network and terminating equipment. Assuming that you tested under varying call load, network conditions, and times of day, if the quality scores were comparable to PSTN calls, you could declare your speech quality assurance job done. But how do you deal with a VoIP call that yields less than perfect quality? The answer is to start by evaluating the quality of service of the data network separately from the quality performance of the terminating VoIP gateways or IP phone devices.