Per-block lambda tuning

[kornelski]

I'm experimenting with varying quality within an image (e.g. lower quality in dark areas, high-quality DC in smooth areas, tuned quality based on feedback from visual metrics, opacity if JPEG is masked, etc.)

Do you think mozjpeg should provide API for this? What should it look like?

For me it'd be useful to have per-block, per-component multiplier for lambda and lambda_dc parameters in quantize_trellis. WDYT?

[fbossen]

Currently per-block lambda and lambda_dc parameters are computed within quantize_trellis(). One question for you would be what information do you rely on to compute adjusted lambda values? Would you like to be able to provide a set of lambda adjustments externally (like you can provide a quantization table)?

[kornelski]

Yes, I'd like to provide adjustments to the encoder from an external source (via a function call or arrays in cinfo struct).

There are cases in which fine quality control is useful, but which are out of scope for the libjpeg, e.g.ZorroSVG takes JPEG and applies alpha channel mask to it sent out of band. In such case JPEG quality should ideally be set proportional to opacity of each block (compress semitransparent blocks harder).

I don't want to provide absolute lambda value, as that's a bit complex to calculate. I only want to tweak it up and down. Ideally I'd provide a simpler quality setting where 0 = lowest quality (all ACs 0?) and maximum = usual quality.

I've tried something like:

lambda_dc *= cinfo->per_block_quality[component][row][2*col] / 255.0;
lambda *= cinfo->per_block_quality[component][row][2*col + 1] / 255.0;

and it seems to do the trick (i.e. two bytes per block, one for DC one for ACs. 0 = lowest quality, 255 = normal quality).

[fbossen]

The general idea sounds reasonable to me. Until we have resolved ABI issues (see #85) it may be ok to add data structures to the cinfo data structure. I would suggest keeping DC and AC adjustments in separate arrays. As for the scale that you use (division by 255) it may be worthwhile to make it generic and specify it in a separate field (one value for entire image). That would enable one to tweak things up or down instead of just down.

[kornelski]

In lambda *= x, the x is very non-linear. To get sort-of perceptually linear quality change in JPEG q=50 I have to use lambda *= x*x*x, but in JPEG q=95 even lambda *= x*x*x*x*x gives barely any quality gradient.

Do you know why is that? What formula should I use to convert quality % to lambda multiplier?

[fbossen]

I wouldn't expect trellis to have much of a visible impact at q=95. Most of the gain from trellis generally comes from zeroing coefficients that would normally quantize to +/- 1. At q=95 I would expect many coefficients to not be affected by the trellis process unless you are very aggressive with lambda. Ideally one would want to change the quantization step instead of just the lambda on a per-block basis. Unfortunately such a feature is not supported by the JPEG specification.

[kornelski]

Indeed. I'm looking for that "unless you are very aggressive with lambda" bit :)

For DC trellis I've had to add more candidates. I'll also experiment with choosing candidates differently (e.g. always try 0 and maybe symbols with fewer bits instead of just trying q ±1).