Thumbnails have wrong colors for Dolby Vision files

[Keith94]

With Dolby Vision files you have to use --vo=gpu-next to get proper colors. However, thumbnails are not color corrected like the actual video, so you get the purple and green hues in the thumbnail.

Reference: https://github.com/mpv-player/mpv/issues/7326

[Demetter]

Same with HDR. Thumbnails look washed out.

[hooke007]

vo_lavc lacks the full support for HDR. Nothing we could do here.

[Keith94]

I wonder if the thumbnail can automatically use the default video renderer as well.

[hooke007]

It never used renderer. Just decoding and encoding.

[po5]

The only way that I know of which would let us do tonemapped thumbnails would be to take screenshots. Sadly screenshot-raw isn't accessible by lua scripts so we'd need to use screenshot-to-file. Would likely require a lot of changes, but I am looking into it.

[po5]

Found a good compromise in 6c99377d1c7ce022f707cb69b8cd25ce5d4fc6fd.
It will do good-enough tonemapping on HDR files (and DV files with an HDR fallback?).
The algorithm is copied from your mpv config, if it's available in ffmpeg. Otherwise it falls back to hable.

It's enabled for videos with bt.2020 primaries, is that a good enough check or are there more subtleties?

ffmpeg needs to be compiled with --enable-libzimg. Will automatically disable itself if the filter fails.
If you know of a way to detect (without spawning a process) which lavfi filters are available, that'd be nice to have.

ffmpeg wizards, is there a way to approximate DV tonemapping too, without a GPU? libplacebo can be used but I don't think -init_hw_device is exposed in any way, so it fails.

[hooke007]

ffmpeg‘s tonemapping values doesn't fully match mpv’s ones. https://ffmpeg.org/ffmpeg-filters.html#Options-3

[hooke007]

Currently property tone-mapping will always return auto unless user manually set it to one of the algorithms.

[po5]

Yup, and nothing that can be done about that. This commit falls back to hable if you configured an unsupported tonemapping algorithm.
I don't think it fully matches libplacebo's tonemapping even with a matching algorithm, ffmpeg's implementation does the minimum.

Currently property tone-mapping will always return auto unless user manually set it to one of the algorithms.

Have you tried it? properties["tone-mapping"] correctly returns hable for my setup. Maybe you didn't see the observe_property line?

[hooke007]

I’m talking about mpv. It returns hable because there is no auto in ffmpeg.

[po5]

Nope you're wrong, please try it. Please try it. It works. Please. Try setting a non-hable option in mpv. It works.

[hooke007]

Try setting a non-hable option in mpv.

https://github.com/po5/thumbfast/issues/84#issuecomment-1546599675

In most cases, mpv will pick bt2390 for hdr, but mpv’s propertytone-mapping will not change its value.

[po5]

Have you tried it? properties["tone-mapping"] correctly returns hable for my setup. Maybe you didn't see the observe_property line?

Please try it, I beg you. properties is populated by your mpv options. It does not come from the fallback. If you explicitly set the algorithm from mpv, it will be taken into account. I don't have access to the final value of auto when that's set as the mpv value. That's too bad.

In most cases, mpv will pick bt2390 for hdr, but mpv’s propertytone-mapping will not change its value.

So that's irrelevant since bt2390 is not supported in ffmpeg.

[hooke007]

No,

I don't have access to the final value of auto when that's set as the mpv value.

which was what I’m talking about.

[po5]

Can it? This is what the performance page of stats.lua uses, but I don't see my tonemapping (even explicit) anywhere.
I get the same values by printing vo-passes myself.

Seems to be the "color conversion" step. Fresh config:

I see that it's available when vo=gpu-next. I guess we can check for it...

[hooke007]

no. and prop vo-passes is not available for all vo

[po5]

Now resolves auto when possible, and allows runtime changes.
I made sure it only fetches vo-passes when necessary.

[po5]

Very cool, will play around with it.

[po5]

It does work but it's slow, to the point of not being useful for generating thumbnails on the fly, at 1-2 full seconds between seeks.
I can only see a use for it if we add support for generating thumbnails in the background, though that's tricky to do with our current model.

Couldn't get hwmap/hwupload/hwdownload to work, either because I don't know how to do it or because it's impossible.
Here's hoping there is/will be a way to tonemap on the gpu from our thumbnailer process.

Sticking to the wrong-but-not-too-slow tonemap filter for now.
https://github.com/mpv-player/mpv/issues/10678 someday.