9 » IC Electronic information » Category: H

IC Electronic information

H.264 hardware decoding error correction in the implementation of collaborative systems

In Electronic Infomation Category: H | on April 22,2011

1 Introduction

H.264 video compression standard with high compression efficiency and LM2766M6 datasheet and good network support, and LM2766M6 price and is suitable for wireless multimedia and LM2766M6 suppliers and Internet-based applications. In the process of information transmission, noise is inevitable. H.264 itself has a very good anti-noise technology, such as SPS, PPS and the separate image data package, a frame can have multiple Slice, flexible macroblock order (FMO) and so on. But there are also many cases the error can not be completely repaired. H.264 high compression ratio (7 to 50 times), which means the image has been greatly eliminate redundancy, from the rest of the image stream is very difficult to recover. In addition, H.264 compression standard uses variable length coding CAVLC and CABAC, once the error in these areas, the decoder will not be able to determine the next variable length codes starting position, resulting in the spread of the error.

The hardware side, the decoding process of each part has its fixed structure to achieve, if not timely detection of errors, can cause such as memory overflow error look-up table, the state machine into an infinite loop and so on. Decode the image dislocation or deformation, more serious when the whole decoder will stop running.

So adding error correction decoder, early to find that the noise stream and the decoder back to the normal state is very significant and very necessary.

Decoder of this project is based on the ASIC SoC solution, it has a high-speed, low power and low cost characteristics. Adding error correction decoder can improve the applicability and stability, on the other hand, should try to reduce the error correction module of the original chip area and speed.

2 stream structure and error detection

H.264 standard compressed stream with a strict format. Public information is extracted as the SPS (sequence parameter set), PPS (picture parameter set) a separate package, with the pixel information separated. SPS and the PPS is extremely important stream of information, packaged them individually in the case of transmission times less well to send, which to some extent, enhanced transmission stream interference. Each package is called a NAL unit, according to NAL type, each packet of data in accordance with the provisions of the agreement closely arranged in the order of syntactic elements. This correction provides two facilities: (1) Each NAL errors will not spread, that is, if the current NAL detected an error and is unable to determine where the error termination, then the current packet can be discarded until the next NAL began; (2) a lot of syntax elements have their fixed range. Decoding the detected elements beyond the limit value is a very useful approach, but also early detection of errors, to avoid the codec to the abnormal state.

H.264 video stream consists of five levels of information: sequence layer, picture layer, slice layer, macroblock layer, the sub-macro block layer, respectively, and NAL types of SPS, PPS, Slice (including titles, macroblock , sub-macroblock information) corresponding to.

Codec processing macroblock is the basic unit is the basic unit for error repair. H.264 video stream structure shown in Figure 1.

Figure 1 H.264 bitstream structure

Several levels above, each layer has its specific syntactic elements of the structure, and the corresponding syntactic approach, the error detection function is based on this realization. Errors can be detected at present based on the following categories:

(1) reserved-bit errors. This is most likely to detect errors. In the H.264 protocol provides a number of reserved bits, such as the NAL initial forbidden_zero_bit, SPS in reserved_zero_4bits and so on. Read into the decoder bit stream in order to determine when the value of these reserved bits are correct, if wrong, they should discard the current package or marking errors for further processing.

(2) syntax element value is not specified range. Most of the syntactic elements, especially in the SPS and the PPS, has a specified range. If the maximum number of frames used to calculate the log2_max_frame_num_minus4 must be between 0 to 12, if the value read out of this range, which can determine the current reading into the code stream for errors.

(3) conflicts with the syntax element. There are some syntactic elements read a wrong value, but still in the correct range, just by the current statement is not detected. This time we should use the correlation between syntax elements to determine the reasonableness of the value of reading. For example, to specify the current PPS in the reference image sequence parameters of the message that the PPS-> seq_parAMEter_set_id. If the current reference to the SPS, the SPS received ID at all does not exist, you can determine the current PPS SPS or a problem with one of the parties; for slice_type of I, B, P types, mb_type the look-up table method is also different, If I slice appeared inter prediction macroblock type, you can also determine the error.

These methods are can determine the value of syntax element itself out, you can read into the stream, while processing, that in dealing with PPS, SPS was added to determine the appropriate statement and will not cause too much of the original decoder effects.

Not all errors can stream directly through the syntax element values ??to judge them. The value of a number of syntactic elements and will affect the decoding method to be reused, so that some errors can be found in the decoding process, such as:

(4) cited reference does not exist. One way to realize H.264 compression is the use of intra and inter prediction, such as the use of 4 16 16 intra prediction mode and 9 4 4 intra prediction mode. Various modes of the requirements of the surrounding macro blocks are different, 16 16 intra prediction mode of block required for the level of the adjacent macro block information shown in Figure 2. If the current macro block X with the level of intra prediction, then the macro block A must be available. If the current macroblock in the first line, then this prediction method is wrong. In inter prediction, the reference block to specify the number where the reference image and position, if there is no reference to the current number of the frame or reference point to the location of the block beyond the bounds of the image to explain the current reference is incorrect. As macroblock using variable length coding of information and there is no specific symbol segmentation, if errors are found, then the data should be started until the next NAL to be discarded.

Figure 2 Intra 16 16 luminance component level of predictive models

(5) look-up table without the corresponding value. CAVLC and CABAC encoded data is difficult because of its variable code length data separated out wrong, but the decoding of these data contains a large number of look-up table operation, very conducive to early detection of errors.

(6) other abnormalities. If a vacancy arises in the queue reference, this time can only judge a frame or several frames in front of a memory management errors, management enable syntax elements daptive_ref_pic_marking_mode_flag was incorrectly set to 1, or the specific type of operation errors. This situation can not decode the syntax elements in the time immediately determine the error (values ??within the normal range and the queue is not unusual), although in time debugging can manually back into the reference frame to the queue, check out the situation, but for real-time decoder for this is meaningless. This time do not give up the current data in the NAL, as long as the information will be near the reference frame can be copied to the reference queue.

Error detection method above is the general summary. Because the noise is random, errors may occur in the decoding process of any one place, only by commissioning a large number of streams in order to achieve a certain degree of fault coverage, so that the decoder has better adaptability.

3 hardware and software error correction in the realization of collaborative system Although the official

H.264 JM reference software is given a fairly good error concealment methods, but given the error correction part to minimize the impact on the original hardware, we use 16 16 intra prediction based on the patch approach, the principle and the 16 16 intra prediction decoding method similar. If you find that starting from a macro block errors, the decoder will determine the presence of neighboring macroblocks and projections for the current macroblock to choose the best forecasting model, through the neighboring macroblock pixel value on the boundary of the current macro block patch and the subsequent macroblock until the end of the current Slice.

The use of hardware and software decoder SoC collaborative implementations. Division of the functions, SPS / PPS / Slice header parsing and other branches of more higher by the flexibility of the software part of the implementation, the macroblock entropy decoding and prediction of a large number of complex operations required by the hardware module. Hardware is divided into two parts front and back end, front end parts, including entropy decoding unit, IQ / IDCT, motion compensation, including back-end (MC) and filter modules.

CPU with each module, the module uses wishbONe bus communication between the front and back end processing unit of a data channel between the another, to share the wishbone bus overhead, the structure shown in Figure 3. CPU input stream of a preliminary treatment, such as the extracted image size, frame type and other information, and then the processed data into the hardware front-end part of the front-end processing of the output is sent to recover the pixel motion compensation module information and remove the block effect.

Hardware and software error detection is done jointly. Software [5] analytic SPS, PPS, Slice head solved at the same time to determine whether the value of the various syntactic elements in a reasonable, if there is an error then send signals to the hardware via the bus HasErr_soft. For example, in the analytic function of PPS, the decoded unsigned index Columbus (ue) are used to indicate the current PPS SPS ID called by the syntax elements seq_parameter_set_id, and then determine whether the decoder received the ID label of the SPS, if no such SPS , then interrupt the current PPS decoding, return to the previous level of function. The parameters of the current contents of PPS PPS copy obtained by the other. That is to do the following changes in the software section:

Read_new_slice () / / read into a NAL unit



Switch (nalu-> nal_unit_type) / / determine the NAL type




ProcessPPS (nalu); / / PPS Analysis





ProcessPPS (NALU_t Wei nalu)



Pps-> seq_parameter_set_id = ue_v (...);

/ / Read the current PPS corresponding SPS_id

If (pps-> seq_parameter_set_id invalid)

/ / If read in SPS_id not available

... / / Copy the contents of the previous PPS

HasErr = 1; / / Error flag 1

/ / (To send the signal through the bus hardware)



Responsible for hardware and sub-macroblock layer decoding of the macro block layer, to find prediction models, entropy decoding error. If the hardware decoding process to detect errors, hardware to software sent HasErr_hw signal. For example, intra prediction, the need to determine the availability of neighboring macroblocks. Intra prediction mode in a statement read by adding the following after the judge:

The correction process is controlled by software, repair process data directly from the software back to hardware. Hardware error detection signal to the software delivery HasErr, the software will do the appropriate data processing, the hardware signals sent Err_Processing and repair program based on erroneous data processing instead of the front portion of the back-end decoder to provide data, as shown in Figure 3, dashed line .

Figure 3 H.264 decoder chart

In the repair process, due to the prediction based on spatial continuity, so you can use the original part of intra prediction hardware. However, some of the original error signal after the appropriate treatment needs to be done, otherwise control access to some of some of the state led to the abnormal state can not continue the normal operation of the hardware.

4 results

The first experiment on the C model of PC, modify and improve the use of hardware testing platform development board Xilinx Virtex4 series. Part decoder software written using the C language, Linux under the gcc compiler, the generated binaries to run on based on the open nuclear OPENRISC1200 the CPU; hardware using Verilog HDL description. Testing the input stream provided for the ITU-T reference sequence [2], adding 11 fixed noise sample. Experiments show that the decoder can successfully decode the stream of noise ratio has been greatly enhanced image playback smooth, a lot of damage to the place to get a better fix, but some can see visible signs of repair.

5 Conclusion

Add the decoder error checking and repair functions, improve the decoders ability to deal with impaired stream, the decoder can therefore use a more complex environment, stability is improved. H.264 video compression standard itself has some support for error concealment techniques, such as SPS and can be repeatedly transmitted PPS, more than Slice, FMO and so on. SPS and PPS repeated transmission transmission efficiency to reduce the cost of the receiver to obtain a higher accuracy rate, which in some environments is essential. Slice and FMO more than a certain extent, increased the complexity of encoding and decoding, but it makes more macroblocks surrounding the macroblock information can be used for prediction and damage repair. In addition, for purposes of repair methods, but also inter-related and the other based on intra prediction method of interpolation. The main objective of this study is to enable the decoder to successfully decode and try to re-use of existing hardware unit to reduce the error correction caused by the addition of hardware has increased. 16 16 intra prediction using the method of repair, the image quality when compared to unpatched greatly improved, but more obvious in some places still have the patch trace. Image patches in search of better ways to repair the image after the original image as close as possible to reduce the repair can significantly distinguish traces of these is the need to continue to study the contents.

LM2766M6 datasheetLM2766M6 suppliersLM2766M6 Price

All right © 2010-2016 Certificate