Add support for random dithering (in addition to ordered dithering)

[GoogleCodeExporter]

There is a bug with all YUV-encoded h264 video in LAV Video Decoder, where the 
RGB result has subtle vertical stripes on every other pixel column.  This 
doesn't happen with the same videos in ffdshow.  It's very hard to notice with 
any high-res video, without upscaling, or with a low-contrast monitor, but with 
upscaled retro game video, it's distracting and easy to notice.  I have all 
output formats except RGB24 and RGB32 disabled, so LAV Video Decoder is doing 
the colorspace conversion.  For the record, the RGB h264 encodes of the same 
footage don't show this striping.  An older version of LAV Video Decoder (don't 
remember which) had this problem when decoding RGB encodes, but when I updated, 
it was fixed, but not for YUV encodes.

Here's the same frame of the YUV 4:4:4 encode with LAV and ffdshow, upscaled to 
fit my screen resolution (my normal viewing behavior in MPC-HC).

LAV Video Decoder 0.45: http://dl.dropbox.com/u/35695760/00000231.png

ffdshow-tryouts r4291: http://dl.dropbox.com/u/35695760/00000232.png

I have attached the lossless RGB and YUV 4:4:4 versions of the same game 
footage.  The RGB encode looks perfect with LAV, but the 4:4:4 shows the 
striping.

Original issue reported on code.google.com by lexlexlex@gmail.com on 31 Jan 2012 at 9:06

Attachments:

• sonictripletrouble-zeupar-444.mkv
• sonictripletrouble-zeupar-rgb.mkv

[GoogleCodeExporter]

This affects all of the officially-published encodes of TASes from 
http://tasvideos.org , so fixing this would improve video playback quality for 
a huge community of gamers (at least, those who have switched to LAV Video 
Decoder as their primary decoder).  :)

Original comment by lexlexlex@gmail.com on 31 Jan 2012 at 9:27

[GoogleCodeExporter]

I can reproduce the problem, will look at it soon.

Original comment by h.lepp...@gmail.com on 31 Jan 2012 at 1:36

• Changed title: Vertical stripes when RGB converter is in use
• Changed state: Accepted
• Added labels: Component-ffmpeg, Filter-Video, Priority-High, Usability
• Removed labels: Priority-Medium

[GoogleCodeExporter]

OK, i found the culprit. The dithering pattern was incorrectly being applied 
because the RGB converter processes two lines at once, causing those obvious 
line patterns instead of a checkerboard pattern which just blends into the 
background.

The kind of dithering i use (ordered dithering) will sadly always cause some 
kind of pattern to be visible, however its really hard to see when its working 
OK.

I might add an option for a random dithering pattern in the future (like 
ffdshow uses), for people that really hate the fixed pattern.
This is really only visible on big same-color surfaces, which i guess these 
game videos have. ;)

Original comment by h.lepp...@gmail.com on 31 Jan 2012 at 4:56

[GoogleCodeExporter]

If I have a solid color in a lossless video with a YUV 4:4:4 color space, I 
would expect it to all come out the same color.  If that's not the case, I 
don't understand the interpretation of "lossless" here.

Thank you for your hard work, nonetheless.  However, I hope there's some 
solution possible where the solid color is uniform.

Original comment by lexlexlex@gmail.com on 31 Jan 2012 at 5:00

[GoogleCodeExporter]

Converting RGB into YUV (and then back) is not lossless, thats your whole 
problem right there. :)

Original comment by h.lepp...@gmail.com on 31 Jan 2012 at 5:01

[GoogleCodeExporter]

I just figured each pixel had its own definite color in YUV 4:4:4, so if each 
pixel was converted the same way to RGB, it would be the same color as its 
neighbors which are stored with the same data.  I realize the conversion is not 
lossless color-accuracy wise, but I thought it was lossless geometry-wise.

Original comment by lexlexlex@gmail.com on 31 Jan 2012 at 5:09

[GoogleCodeExporter]

ffdshow certainly makes it seem so, anyway.  The solid color is completely 
solid in ffdshow, and that's with all output formats except RGB disabled.

Original comment by lexlexlex@gmail.com on 31 Jan 2012 at 5:11

[GoogleCodeExporter]

The conversion from YUV to RGB (and the other way around) results in floating 
point data, which you somehow have to convert back to 8-bit RGB. There is 
basically two ways to do that - rounding or dithering.

Rounding has the advantage that single color planes remain the same color, but 
it has the disadvantage that slight gradients might be converted into the same, 
causing an effect known as banding.
Dithering on the other hand will try to spread the rounding error among 
neighboring pixels, that way banding is avoided. It does however come at the 
cost that such special videos as yours here will have the possibility of 
actually making the dithering pattern visible.

There are several dithering methods used, one is ordered dithering (which uses 
a fixed pattern), another is random dithering (with a completely random 
pattern). Both have their advantages and disadvantages, and its really a matter 
of taste. ffdshow uses random dithering, i use ordered dithering.

That there were these lines was definitely a bug, it should be more of a 
checkerboard pattern. If its done properly (with the bug fixed), the upscaling 
process will mask the pattern, making it much less visible.

To avoid these issues altogether, the best way is to upscale before dithering, 
that way the dithering pattern don't get upscaled as well, and the result 
should be *much* better.

Original comment by h.lepp...@gmail.com on 31 Jan 2012 at 5:19

[GoogleCodeExporter]

Excellent explanation.  :)

I'm glad my RGB videos play back with LAV Video Decoder, anyway (and LAV was 
the first to be able to do so in a filter!).

I will stick with RGB, in that case.  The extra file size (normally; not with 
these samples for some reason O.o) is worth it for total visual losslessness.

Original comment by lexlexlex@gmail.com on 31 Jan 2012 at 5:26

[GoogleCodeExporter]

I don't see the pattern with my changes anymore, but if i take a screenshot and 
upscale it even further, i can see it again.

It may be more obvious if you have a cheap LCD display, its quite common for 
them to only have a 6-bit panel, and if they use ordered dithering as well to 
get to that 6-bit, it'll be bad. Dithering a dithered image causes the pattern 
to amplify.

I'll add random dithering (like ffdshow) in the future, which should resolve 
those problems.
For the record, on a screenshot taken from ffdshow, i can still see 
differences, but because its completely random, it doesn't build up to a 
geometrical pattern (which the human eye can find easier then random pixel 
errors)

Original comment by h.lepp...@gmail.com on 31 Jan 2012 at 5:30

[GoogleCodeExporter]

Original comment by h.lepp...@gmail.com on 31 Jan 2012 at 5:32

• Changed title: Add support for random dithering (in addition to ordered dithering)
• Added labels: Priority-Medium, Type-Enhancement
• Removed labels: Priority-High, Type-Defect

[GoogleCodeExporter]

For the record, ffdshow must do that rounding you were talking about, 
considering I can actually use MS Paint's fill tool on the sky in my screenshot 
and it colors the entire sky, proving it's not just a perception of solidity.

My panel is 8-bit-per-channel RGB, luckily, so I don't see an amplified effect. 
 It's subtle, but distracting once it catches my eye.

Original comment by lexlexlex@gmail.com on 31 Jan 2012 at 5:35

[GoogleCodeExporter]

Make sure you actually have dithering in ffdshow activated... ;)

Attached image is ffdshow with dithering, i randomly clicked around in the 
image using the fill tool. You can see its random dithering pattern, although 
its not as random as i thought it would be.

The second is LAVs dithering after i fixed the line problem.

Original comment by h.lepp...@gmail.com on 31 Jan 2012 at 5:41

Attachments:

• ffdshow-dither.png
• lav-dither.png

[GoogleCodeExporter]

oddly, dithering is enabled: http://dl.dropbox.com/u/35695760/00000237.png

OHHHHH.  I just realized something.  The fill tool I claimed was MS Paint was 
actually in IrfanView.  I was assuming that it was the same mundane fill tool, 
but it actually fills "close" colors.  If I actually go into MS Paint, the fill 
tool doesn't "leak", and attempting to fill in the SEGA logo results in 
patterns like that.

Yeah, that doesn't look as random as one might think.  Very interesting.  It 
definitely looks very difficult to notice in ffdshow, and surely similar with 
your now-fixed dithering in LAV Video.  :)  Thanks again for your excellent 
work!

Original comment by lexlexlex@gmail.com on 31 Jan 2012 at 5:48

[GoogleCodeExporter]

I downloaded your beta build from the doom9 forum which has the fix.  The 
dithering pattern was still quite visible in LAV, but still seemingly invisible 
in ffdshow.  I did some tests to figure out why.

I did some tests on screenshots of the cyan-like sky in my test video in MS 
Paint to check colors.  I found that LAV's dithering only uses 4 colors, while 
ffdshow's uses at least 8.  Here are some colors I found from my eyedropper 
tool tests.

lav: checkerboard pattern of these (and a few more)
{0,193,227}
{1,194,228}

ffdshow: random pattern of these (and a bunch more)
{0,194,227}
{0,193,228}
{0,193,227}

I used IrfanView's color count feature to count unique colors in sections of 
solid sky copied from my screenshots.  In the LAV screenshot, it found 4 unique 
colors.  In the ffdshow screenshot, it found 8.

I'm only posting this so that when you get around to implementing random 
dithering, you consider using a broader range of colors than your ordered 
dithering pattern does, with more minor differences between neighboring pixel 
colors.  I'm not 100% sure why the random dithering pattern with that many 
different colors looks so solid and invisible, but it's probably due to 
smoother gradients between neighboring pixels.  In your ordered dithering 
pattern, neighboring pixels had relatively major differences (red, green, and 
blue all 1 off, rather than just 1 channel difference), making the contrast 
plain to me.

Sorry if I seem bothersome.  I realize you're doing this amazing work for free. 
 Thank you for your time.  :)  Also, I greatly appreciate your explanation on 
YUV -> RGB color conversion using dithering to handle the fractional color 
numbers.

Original comment by lexlexlex@gmail.com on 4 Feb 2012 at 2:34

[GoogleCodeExporter]

For the record, the correct original solid color for the sky there is 
{0,192,224}, as seen in the rgb lossless video, not that that could be properly 
gleaned from any YUV 4:4:4 10-bit video.  What's crazy to me is that YUV 4:4:4 
10-bit (as used in this particular sample) is supposed to have much better 
conversion accuracy than YUV 4:4:4 8-bit, and yet the conversion (probably 
mainly in the encoding rather than the decoding) is still so inaccurate (blue 
value is up to 4 higher than the original).  Maybe I'll have to learn more 
about more more accurate color space conversion.

Original comment by lexlexlex@gmail.com on 4 Feb 2012 at 3:02

[GoogleCodeExporter]

I just implemented random dithering (very random, more so then ffdshow).
I couldn't find a spot where the fill tool would fill a significant area 
anymore. ;)

Give this a try:
http://files.1f0.de/lavf/LAVFilters-0.45-49-ga35b4a6.zip

Before it goes final, i'll be adding an option to switch between ordered and 
random, still have to think how to implement that.

Original comment by h.lepp...@gmail.com on 4 Feb 2012 at 3:50

[GoogleCodeExporter]

By the way, the problem with "few colors" is a by-product of the ordered 
dithering, because it really only has two patterns, (0, 1, 0, 1) and (1, 0, 1, 
0), so even if i use these alternating, with 3 color channels the chance is 
high that there will always be 2 changes. Random solves that problem just by 
being random. :)

Original comment by h.lepp...@gmail.com on 4 Feb 2012 at 3:51

[GoogleCodeExporter]

Yes, now it's very difficult to see.  I like it a lot.  :)  Thanks again for 
your hard work!

Original comment by lexlexlex@gmail.com on 4 Feb 2012 at 3:56

[GoogleCodeExporter]

All converters now support both ordered and random dithering, and an option was 
implemented.

This issue is resolved.

Original comment by h.lepp...@gmail.com on 5 Feb 2012 at 9:28

• Changed state: Fixed