Awfully Bad Rounding Errors when Decoding Lossy JXL (Dithering Needed!)

[Matt-Gore]

The decoder of libjxl produces ugly rounding errors when decoding a lossy 8 bpc JXL, which results in visible banding.

We will take those two images as examples: (full resolution versions found here and there)

This is how (a part) of the image looks after encoding it "visually lossless" (-d 1) and decoding it back to 8 bpc pixels:

Notice how wrong it looks? Here's an exaggerated view of the situation (that could happen on a cheap display or to people sitting in a dark room):

You might ask yourself how this visual bug could happen, because if you look at the lossless original picture (which is also just 8 bpc), everything seems fine. But don't worry, we don't need to throw away JXL since it all can be fixed! So, JPEG XL uses DCTs for lossy images (by default) and very small details like the patterns of noise are not considered worthy or important to store when using "visually lossless" quality. It thus leads to simpler DCTs, however, they can still represent a smooth gradient. The actual issue happens at a late stage of decoding. JPEG XL uses 32 Bit floating point precision to operate as accurate as possible but when outputting to 8 bpc pixels again, it messes up by just dropping essential information and doing very inaccurate and unnecessarily simple rounding. Instead, quantization of the float values should utilize a brand new, still rather unknown discovery in digital signal processing called DITHERING! Jokes aside, but a quick and easy improvement with a huge benefit would be doing something like applying 4x4 ordered dither. (This would be better than just having to add [photon] noise.)

That's how great it would look with dither:

Which is orders of magnitudes better than what is currently been done. And causes no noticeable banding:

Here is the other example (same picture order):

And here is an artificial file to play around for yourself.

This feature is also beneficial when the JXL stores a (lossless) 10 or 12 bpc image while the client requests an 8 bpc output. Optionally there could be a parameter to disable dithering, in case some weird person doesn't want smooth gradients. And since it is possible to access the 32 Bit floats, applications with a focus on image quality rather than performance can also use something more fancy like error diffusion.

Old GitLab issue: https://gitlab.com/wg1/jpeg-xl/-/issues/136

[Visual Bug] | [Essential Improvement] | [Let VeLuca Fix]

[jonsneyers]

I think that whenever we produce integer output buffers in the API (i.e. uint8 or uint16), we should do some dithering when quantizing our internal floats to integers. Applications will likely also need to implement dithering themselves, if they want to resize the image before showing it, but in any case the effect of quantizing to integers will be less problematic if we apply dithering. It can be a simple ordered dither that shouldn't have any noticeable impact on performance.

[xiota]

Since the images are so dark, maybe too much information is being thrown away by the encoder. In images with very narrow dynamic range, subtle differences may be more important to keep. When I use target-size to match the webp file size, the banding is not apparent with d=0.031.

Just adding noise seems helpful, and it looks like the webp image may have some added noise. The strips in sequence are webp original (1.7MB), JXL (d=1, 49KB), JXL+noise. On the left, auto levels was applied. On the right, gamma was applied to make banding easier to see.

[jonsneyers]

Just decoding to 16-bit and doing a dithered quantization to 8-bit also works. The information is already available in the jxl, it's just that it gets thrown away at the moment we convert to uint8.

[xiota]

Banding is still apparent in the gimp plugin with the final float→int conversion disabled. I also made it add layers after some naive data manipulation to see what dithering would look like. I used this matrix I found online (Image Dithering: Eleven Algorithms and Source Code) because the sample images looked relatively nicer than the other methods:

Sierra Dithering (1/32)
       X   5   3
2   4  5   4   2
    2  3   2

Maybe I'm applying dithering wrong, but just adding noise seems better to reduce the appearance of banding.

JXL with d=1(float), random noise added, sierra dithering. Auto levels on the left. Gamma 1.5 on the right to make it easier to see the banding.