DVD Chroma Upsampling* Error, Simplified
In the normal playback of DVD, tiny horizontal streaks of incorrect coloration often occur with most players. These streaks occur along the top and bottom edges of color patches and also show up as indentations along diagonal color boundaries giving a serrated or comb like edge. Usually, the viewer does not notice unless someone else points it out and then that viewer seems to see it more and more. While the viewer expects occasional speckles or snow in TV broadcasts, DVD is expected to give a clearer picture, and imperfections are then more noticeable.
In order not to have the chroma upsampling error, the DVD player requires two data decoding strategies, one for film source and the other for live video source. In practice, most players have only the latter.
Chroma Upsampling Error (CUE)
Interlaced Chroma Problem (ICP)
What is 4:2:0? 4:4:4?
The Chroma Upsampling Error (CUE)
The chroma upsampling* error, or "chroma bug", occurs because a DVD (and also the current U.S. digital TV standard) only provides color for every other scan line. In other words, for film source material, scan lines 1 and 2 share the same color (it should be the average of the two scan lines), scan lines 3 and 4 share the same color, and so on, although light/dark detail may be unique for each line.
In technical terms the color resolution is half the luminance resolution. This compromise was made to make the entire program fit on a 5 inch disk or be transmittable in the broadcast channel bandwidth allowed. The human eye is not as sensitive to minute errors in coloration compared with fine light and dark detail.
But DVD players always decode the disk data as interlaced. The color for scan lines 1 and 2 is brought out for the odd scan lines and gets applied to scan lines 1 and 3. The color for scan lines 3 and 4 comes out for the even field and is applied to scan lines 2 and 4. So these minute color errors occur. Sometimes these errors are conspicuous, such as where a dark blue patch meets a bright red patch. The light/dark relationships remain correct and unaltered, and the interchanged coloration in this example produces some bright blue where bright red should be as well as the less obtrusive dark red where dark blue should be.
For progressive scan output the player has the opportunity to assemble the entire frame and then apply the color correctly. Unfortunately many players' circuitry still applies the color to the scan lines first (color for scan lines 1 and 2 applied to scan lines 1 and 3) before assembling a complete frame.
The top right diagram (Shared Color) shows how lines of color might be averaged when every two luminance scan lines share the same coloration.
In the second row in the diagram above we see how the color for scan lines 1 and 2 is packaged with luminance scan lines 1 and 3, and so on. If the MPEG decoder obtains the color for both odd and even fields when constructing each field for film source material, it can avoid the chroma upsampling error. (For non-film source, colored motion artifacts may appear, discussed later under "Interlaced Chroma Problem").
In the second row right diagram we can see the effects of using the color as packaged in the second row left diagrams, resulting in interchanging the color for the second and third scan lines and for the fifth and sixth scan lines. This is the chroma upsampling error.
At normal viewing distances, it is not too obtrusive. For this example, stand 15 to 20 feet away.
For 24 fps film source (most DVDs) consecutive interlaced fields repeat the subject matter in a 3-2 pattern. It is almost always possible to construct a complete video frame from what was each film frame, and therefore it is almost always possible to apply color to each scan line without CUE. (Once in awhile an exception occurs at a scene change and only lasts for 1/60'th of a second.)
For live video where each interlaced field could differ from both of its neighbors in terms of subject motion, it is not practical to "fix the chroma upsampling bug". Where there is subject motion, having scan lines 1 and 3 share the same color is actually better and the source material should provide the average of scan lines 1 and 3 for the color information. The progressive scan DVD player needs two formulas for applying color to complete video frames, one for film source, the other for live video source.
For regular (non-progressive) DVD players, the player obtains (decodes) and outputs all of the odd scan lines before obtaining the even scan lines. It is possible but not common for the color for all scan lines to be obtained during the processing of the odd scan lines and again during the processing of the even scan lines for interlaced players also. A progressive player could even regenerate (interlaced) S-video or composite output with CUE fixed for those viewers who do not have a progressive scan TV yet.
CUE is not present in ordinary (analog) TV broadcasts, VHS tapes, or laser disks because here every scan line has its own color.
Undoing the Chroma Upsampling Error
Taken as a whole, the (progressive scan) video frame with the CUE has two scan lines with correct color content followed by two scan lines whose color content has been interchanged followed by two scan lines with correct color and so on alternated. (This is why not every color boundary will have an error in it.)
Once the video comes out of the MPEG decoder, the color has already been assigned to each scan line (or row of pixels) but for DVD the luminance and color data are separate entities. If the color was applied incorrectly, the only way to undo the error is to analyze the video to determine this, and then exchange the color content of the affected lines. This is a complicated process. We do not have proof that, supposing that the incorrectly colored scan lines were 2, 3, 6, 7, 10, 11, 14, 15, 18, 19, and so on in the first frame, they would be in the same places throughout the movie. If not, frame analysis must take place continuously.
Some video equipment (video processors) use filtering to hide CUE. Generically such filtering is a form of "vertical filtering" which by definition commingles adjacent scan line content and reduces vertical resolution (vertical chrominance, or color, resolution in this case.)
Typical result:
Original With CUE Filtered
red red red
red red red
red red orange
red green amber
green red amber
green green lime
green green green
green green
green
In this near worst case example it takes six scan lines or 1.2% of the screen height to transition completely from one color to another; for DVD the maximum color resolution requires two scan lines for the transition. Fortunately CUE does not affect light and dark nuances in the picture and at normal viewing distances, this softening of color resolution is for the most part not obtrusive.
The Interlaced Chroma Problem (ICP)
Non-film or "live" video recorded on DVD also has the rule of every two scan lines sharing the same color.
For interlaced video, experts still speak of a full frame of video that consists of an odd field and the even field following. There gives the usual total of 480 luminance scan lines which for DVD share 240 lines of color. In this discussion we will give only the numbers for NTSC but the same situation also occurs with PAL.
For live video on DVD, each 240 line field stands alone with 120 lines of color. The first line of color still goes with the first two scan lines which are now properly #1 and #3 (#2 and #4 for the even field); the second line of color still goes with the next two scan lines which are #5 and #7, and so on.
The problem has to do with where the color came from.
Many production methods still assume that the 240 lines of color for each 480 scan line frame are evenly spaced, that the first line of color should represent the first two scan lines (#1 from the odd field and the first line of the even field). So for the first interlaced field, it is common to begin by interpolating what the color for "line 1-1/2" should be by taking a 75/25 percent average of scan lines 1 and 3. (For the even field a 25/75% average of scan lines 2 and 4 is used as the starting point to guess at the color for "line 3-1/2".) Sometimes only scan line 1's (line 4's for the even field) color is used as a starting point. The fallacy is that the color has to eventually go back onto scan line 3 (line 2 for the even field.) What then happens is that the color for scan lines 3 and 2 may be profoundly incorrect, resulting in discolored streaks of an appearance similar to the chroma upsampling error discussed earlier.
Nor will accessing the color for both the odd and even fields, used for fixing CUE, fix this problem. Taking for example the line of color weighted for scan line 4 (from the even field) and using it as a better match for scan line 3 (of the odd field) will produce discolored streaks as a motion artifact wherever subject movement is represented from field to field. Even with unweighted averaging, the color for lines 1 and 3 is interpolated from lines 1 and 3, not taken from line 2.
So there is no easy way to eliminate this problem completely. Video production doing a 50/50% average to obtain the color for each shared color scan line will minimize the obtrusiveness of ICP but also result in a softer picture overall.
Good de-interlacers weave material from the next (and/or previous) field (for stationary subject material) and average (interpolate) the two neighboring scan lines to make the line in between (for moving subject material). Generally the color is already applied to each scan line before going into the de-interlacer. ICP can be reduced if this motion adaptiveness can be utilized in selecting parts of the lines of color from an even field to apply to an odd field and vice versa.
Some de-interlacers use only one field's worth of color (120 lines) while still pulling material from the adjacent fields to do the de-interlacing. So one line of color is being shared by four luminance scan lines. This is supposed to reduce thin streaks of the wrong color that resemble the CUE but also makes color changes more blurred.
ICP Examples
To re-iterate, in all cases, every line of color is shared by two scan lines. For non-film video, the first line of color came from and goes back on scan lines 1 and 3. The top diagram shows the common practice of, for every four scan lines, weighting the color for lines 1 and 3 towards the color of line 1 and weighting the color for lines 2 and 4 towards the color of line 4. In the middle we show the scan lines with the shared color put back. When the video is viewed, discolored streaks are quite prominent.
In the filtering example above, the color is modified by taking a 50% weight of what it used to be (second to right diagram) plus a 25% weight from the line above and a 25% weight from the line below.
The lower set of diagrams shows the result when the shared color for each pair of scan lines (1 and 3, 2 and 4, 5 and 7, etc.) is an unweighted average.The scan lines are shown with the shared color already re-applied. Like the result in the diagrams above, this can also be filtered.
The above diagrams represent stationary subject matter. We feel that the discoloration from weaving in lines of color from the next field where motion has occured is obvious enough not to require an example.
What is 4:2:0?
Other discussions of digital video and CUE and ICP refer to terms like 4:2:0 to describe different standards of DVD and HDTV production.
4:2:0 means that for every four luminance pixels across there are two color pixels retained for odd scan lines and none for even scan lines. (The actual color should be averaged for each pair of scan lines and shared as we described above.) We have every 2x2 block of luminance pixels sharing the same color. In practice, half of the color scan lines in 4:2:0 encoding are derived from and packaged with the even fields for interlaced video.
4:2:2 means that for every four luminance pixels across, there are two color pixels for odd scan lines and also two color pixels for even scan lines, in other words each scan line has its own color.
The fact that side by side luminance pixels share the same color in these encoding methods is not relevant to the discussion of CUE and ICP. The equivalent encodings where side by side pixels do not share the same color are 4:4:0 and 4:4:4 respectively.
NTSC DVD uses 4:2:0 encoding on the disk. U.S. ATSC HDTV and DTV format may use either 4:2:0 or 4:1:1.
*Upsampling -- The process of synthesizing additional scan lines or pixels to go between the one that we already have, so as to simulate having gathered more pixels (taken more samples of the original scene) in the first place. Because we have one row of color pixels for every two rows of luminance pixels, we need to create or assemble more rows of color pixels, where the simplest strategy is to just use each row of color pixels twice.
Chroma, or Chrominance -- The color or coloration in video. In almost all video production, during at least one stage, the video exists as a luminance portion which produces a black and white picture, and the chrominance portion. There is a slight technical difference between "chroma" and "chrominance" which is not important in this discussion.
All parts (c) copyright 1997-2003, Allan W. Jayne, Jr. unless otherwise noted or other origin stated.
If you would like to contribute an idea for our web page, please send us an e-mail. Sorry, but due to the volume of e-mail we cannot reply personally to all inquiries.