Automatic chroma subsampling level

[kornelski]

Chroma subsampling gives nice file size savings, but there are cases where it goes horribly wrong, e.g. thin red lines.

It would be great if mozjpeg had heuristics to detect when subsampling is appropriate, taking into account contrast in chroma channels (especially the the one responsible for red color).

[kornelski]

Does anybody know if there was any research done in this area?

I'm wondering whether it's enough to have a threshold based on mean square error with half-resolution image, or does it need to be smarter.

And how much and when luma can mask errors in chroma?

http://infoscience.epfl.ch/record/33957/files/33957-SüsstrunkW04.pdf — has measure of "colorfulness" lost due to subsampling

[jrmsmith]

Have you checked tis paper: PSNR-based optimization of JPEG baseline compression on color images?

On the plus side the paper is 10 years old and I haven't found the patent related to this paper.

[danielgtaylor]

I had to expose a chroma subsampling command line parameter recently and honestly it's kind of a pain to deal with. I'd love to see this be automatic. :+1:

[kornelski]

@jrmsmith That paper assumes distortion of subsampled image is just MSE of RGB channels, so it evaluates MSE(1x1 compression) vs MSE(2x2 compression) + MSE(scaling).

I think it could be improved by looking at MSE of Cb and Cr, with high weight on Cr (maybe even as high as R and B weights used for grayscale conversion)

The paper has a very interesting case: their algorithm chose to use subsampling on Goldhill

It's interesting, because in that case subsampling does make the red lines in the image look bad! But extra bits help textures on buildings look much better, so it's really hard to say what should be the correct subsampling.

[brunogm0]

Quality of RGB - YUV - RGB conversion

Nice test of subsampling patterns, but the sensitvity of a few test images could help this feature.

[magicgoose]

I had the idea that one can test the perceptual similarity between the source and the subsampled image (round-trip to subsampled YUV and back), using small overlapped windows (for example, 64x64 blocks with 50% overlap in both dimensions) and taking the worst score.
Then, if the worst score is worse than some threshold, subsampling is not applied.

It's interesting, because in that case subsampling does make the red lines in the image look bad! But extra bits help textures on buildings look much better, so it's really hard to say what should be the correct subsampling.

It should somehow depend on the chosen quality level. At some point, the loss of subsampling becomes bigger than the gain from extra bits.