TIA TIA-1017

Abstract

This document specifies the procedures to test implementations of VMR-WB-compatible variable-rate speech codecs <em>either by meeting the bit-exact implementation, or meeting recommended minimum performance requirements</em>. This speech codec is corresponds to the Service Option 62 described in TIA-1016 [1]. The VMR-WB codec corresponds to Service Option 62 and is used to digitally encode the speech signal for transmission at variable data rates of 13300, 6200, 2700, or 1000 bps. Like some other speech coding standards, this standard provides a bit-exact method of testing the codec for minimum performance. In this optional procedure, a given set of test vectors are input to the test codec, and the output vectors from the test codec must be bit-exact with the output vectors generated from the Software Distribution which is part of this standard. If they are bit-exact, and the test codec complies with Section 2.1.2, the codec passes the minimum performance requirement and no further testing is required. The bit-exact mode of testing, however, is only applicable to codecs whose design conforms in all respects to the algorithmic description of the VMR-WB in TIA-1016 [1]. The bit-exact VMR-WB validation includes specific testing of the OEM VMR-WB to the frame erasure processing characteristics of the standard VMR-WB. Should the candidate VMR-WB differ in any of these components, the codec shall be tested using the objective and subjective tests prescribed by this standard. That is, VMR-WB compliance of a "test codec" can be achieved by either: <br>&#8226; Demonstrating bit-exactness according to the procedure described in Section 3.6, specifically Sections 3.6.5.1 to 3.6.5.4, or <br>&#8226; Following the objective and subjective testing procedures set forth in Sections 2.1 and 2.2 of this standard, using processes additionally defined in Section 3.6, specifically Sections 3.6.5.5 to 3.6.5.8. <br>With the exception of Section 3.6, the remaining text applies only to implementations that do not satisfy the requirement for bit-exactness. <br>Testing the codec is based on two classes of procedures: objective tests and subjective tests. In the event that the test codec fails any of the objective or subjective tests, the test codec fails the compliance test. Objective tests are based upon actual measurements from the speech codec function. Subjective tests are based on listening tests to judge overall speech quality. The minimum subjective requirement for the test codec is based upon the ability of the test codec to demonstrate performance equivalent to or better than that of the VMR-WB floating-point master codec within a fixed allowable statistical error. <br>The purpose of the testing is not only to ensure adequate performance between one manufacturer's encoder and decoder but also that this level of performance is maintained with operation between any pairing of manufacturers' encoders and decoders. This interoperability is a serious issue. Any variation in implementing the exact standard must be avoided if it cannot be ensured that minimum performance levels are met when interoperating with all other manufacturers' equipment meeting the standard. This standard provides a means for measuring performance levels while trying to ensure proper interoperation with other manufacturers' equipment. <br>The issue of interoperation can only be definitively answered by testing all combinations of encoder/decoder pairings. With the number of equipment manufacturers expected to supply equipment, this becomes a prohibitive task; therefore, the objective and subjective tests rely upon the use of a "master codec". The master codec is defined as the floating-point implementation of TIA-1016 [1] written in the C-programming language. The master codec software, described in Sections 3.4 and 3.5, are used as part of the interoperability testing. The master codec is available in an associated Software Distribution as part of TIA-1016 [1]. <br>By convention in this document, the Courier font is used to indicate C-language and other software constructs, such as file and variable names.