search

mpv-player / mpv

🎥 Command line video player
https://mpv.io
Other
28.38k stars 2.91k forks source link

Alternate HDR10 metadata handling #14811

Open mitzsch opened 1 month ago

mitzsch commented 1 month ago

Expected behavior of the wanted feature

Currently, mpv looks at the file and uses the attached HDR10 metadata for image processing frame by frame. This is fine as long it does not change within the file. However, there are some files (namely, those files generated from a UHD BluRay that contain multiple cuts and are stitched together. => seamless branching) that have alternating HDR10 metadata.

For example here: firstframe

... a second later ...

secondframe

As you can see, the HDR metadata changes.

Why is this a problem?

This causes the swapchain (in my case tested with -target-colorspace-hint) to reconfigure and lead to severe framedrops. This is most notably the case when the displays output is frame-matched with the file's video parameters.

For files where the metadata does not change it's not happening, nor does it occur on SDR files. (obviously, as there is no metadata that could change)

This changing metadata behavior is probably violating the HDR10 spec but it would be nice if mpv could handle those files without a hitchup. Commercially available players, like a Blu-ray player, only check the file header/first frame for metadata and do not reconfigure the output while playing.

Alternative behavior of the wanted feature

This is a feature request to the issue I posted over a year ago => https://github.com/mpv-player/mpv/issues/11538 However, this is not a mpv bug, it's just a different file treatment. That's why I opened this feature request - let the user allow how mpv should handle such files.

With a new setting (maybe a libplacebo flag?) the user could change this behavior and configure it as liked.

Thanks

Log File

See issue https://github.com/mpv-player/mpv/issues/11538

Sample Files

No response

quietvoid commented 1 month ago

Commercially available players, like a Blu-ray player, only check the file header/first frame for metadata and do not reconfigure the output while playing.

This isn't true. Some TVs react to changing HDR10 metadata as well, it just does a slow tonemap curve adjustment so that it's not too obvious.

kasper93 commented 1 month ago

See issue https://github.com/mpv-player/mpv/issues/11538

Swapchain shouldn't be reconfigured on HDR metadata change. It has been changed in more recent versions. So if it does, we need a new log.

mitzsch commented 1 month ago

Redid the testing behavior is still the same.

Here are the new logs. mpv-hdr-metadata.log.txt mpv-hdr-metadata_no_swapchaindepth1.log.txt

Right at the spot where the metadata changes this line is printed: [ 25.893][v][vo/gpu-next/libplacebo] New swap chain configuration received from hint: format: R10G10B10A2_UNORM, color space: RGB_FULL_G2084_NONE_P2020.

Note that I initially had the option swapchain-depth=1 option set, as this is needed for Nvidia gpus and recent drivers to get a no-stuttery output. I did a second run without the option, behavior is still the same... frame drops*

If you need the file for testing, let me know.

Some TVs react to changing HDR10 metadata as well, it just does a slow tonemap curve adjustment so that it's not too obvious.

Probably true if you are referring to an internal player. However I did check the HDMI metadata with an HDfury arcana and my BD Player (Panasonic UHD player) and the metadata stayed the same, no change there. *also if the TV receives different metadata it processes it, that's clear, but it's probably not supposed to receive different metadata during playback of one file. My BluRay player only sends the header (first frame?) metadata values, regardless of further changes in the file.

kasper93 commented 1 month ago

Right at the spot where the metadata changes this line is printed: [ 25.893][v][vo/gpu-next/libplacebo] New swap chain configuration received from hint: format: R10G10B10A2_UNORM, color space: RGB_FULL_G2084_NONE_P2020.

This is when it drops frames? From quick look we don't do anything redundant here. We get new format, call IDXGISwapChain4::SetHDRMetaData and IDXGISwapChain4::SetColorSpace1 and nothing more, not even resize is done, because it knows the format actually didn't change. I don't see what we can do here. We are not constrained environment like BD Player, where we could hack static metadata. mpv can be used for livestreams, stitched files, where metadata change is valid and important for proper playback.

Does gpu-apu=vulkan work better? But actually passthrough with Vulkan probably won't work...

mitzsch commented 1 month ago

This is when it drops frames?

Yes, that's where the frames drop.

I don't see what we can do here. We are not constrained environment like BD Player, where we could hack static metadata. mpv can be used for livestreams, stitched files, where metadata change is valid and important for proper playback.

Hmm, that's why I asked for a user-changeable option. Wouldn't it be possible to only use the file header/container metadata if the user wants to? Or only apply this "static" hack on HDR metadata? If you like I can test some code. ^^

kasper93 commented 1 month ago

It is kinda hard, because HDR metadata is demuxed/parsed by ffmpeg and passed as a frame side-data. There is not really good way to ignore it or restore previous one.

mitzsch commented 1 month ago

Hmm, okay I see. Well, that sounds complicated then... sad...

Another note on the "constrained factor" of Blu-ray player. As you said mpv is luckily not constrained but in this case, I rather think that behavior (of not changing metadata) is by design.

Just remembered a chat with a support guy ("Post Production Support Specialist") from Blackmagic Design, where I had a problem exporting HDR media with metadata attached from Resolve. (It was only attached to the first frame) He told me:

As for the HDR10 behavior you mentioned, HDR10 uses static Metadata. As such this would only be applied to the first few frames, and other frames throughout/at the end of the clip.

However, HDR10+ uses dynamic metadata and would have additional metadata for each frame. This should then fit the requirements that you need.

As he said, if metadata should change, someone should rather use HDR10+... So maybe reconsidering HDR10 static metadata behavior (even though complicated) is necessary?