search

GPUOpen-LibrariesAndSDKs / AMF

The Advanced Media Framework (AMF) SDK provides developers with optimal access to AMD devices for multimedia processing
Other
599 stars 149 forks source link

Questions regarding HDR encoding and color conversion #268

Open nyanmisaka opened 3 years ago

nyanmisaka commented 3 years ago

Hello! Seeing that “HDR color conversion in the encoder” is list in the release note of AMF 1.4.21. I have some questions as follows.

  1. If I want to encode HDR video using HEVC encoder. In addition to setting P010 surface and 10-bit property, which fields I need to fill? use _INPUT_ or _OUTPUT_? And what is the difference between these two?
AMF_VIDEO_ENCODER_HEVC_INPUT_COLOR_PROFILE
AMF_VIDEO_ENCODER_HEVC_INPUT_TRANSFER_CHARACTERISTIC
AMF_VIDEO_ENCODER_HEVC_INPUT_COLOR_PRIMARIES
AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA

  1. Are these fields above just to set VUI header into the video? Or doing some color conversion inside the encoder using this capability. AMF_VIDEO_ENCODER_HEVC_CAP_COLOR_CONVERSION

  2. Does the Converter accept P010 surface when using tonemapping options on HDR input? As we know most of the HDR films are encoded in 10-bit. So can I use this option to map HDR(smpte2084) into SDR(bt709) without clipping too much color details? AMF_VIDEO_CONVERTER_TONEMAPPING_AMD

rhutsAMD commented 3 years ago

  1. All of the “INPUT” parameters describe the input surface in the case that it is RGBA. All of the “OUTPUT” parameters describe the HEVC stream and are translated to the VUI header. If you are submitting P010 surfaces, then only “OUTPUT” is needed.

  2. The “OUTPUT” parameters are used to set the VUI header and AMF_VIDEO_ENCODER_HEVC_CAP_COLOR_CONVERSION parameter is used to determine if the HW encoder or shaders will be used for conversion.

  3. The encoder color space conversion is limited where it can currently do SDR to HDR, but cannot do HDR to SDR as you intend. Also, the color converter inside the encoder uses HW so we cannot change parameters for conversion without re-initializing the encoder.

nyanmisaka commented 3 years ago

Thanks for your quick response.

  1. The line AMF_VIDEO_ENCODER_HEVC_OUTPUT_HDR_METADATA was commented out. Should I use INPUT_HDR_METADATA instead? https://github.com/GPUOpen-LibrariesAndSDKs/AMF/blob/3ee61d73b5599dcadbf86621a9f9c2d7e9c05811/amf/public/include/components/VideoEncoderHEVC.h#L208

  2. As you said HDR to SDR is not allowed in the encoder side, does the limitation also exist in AMFVideoConverter component added before the encoder? If so, can we expect to get it in the future?

rhutsAMD commented 3 years ago

Yes, please use AMF_VIDEO_ENCODER_HEVC_INPUT_HDR_METADATA.

We can consider the conversion that you are describing. What use cases do you have in mind and which tone mapping would be preferrable?

nyanmisaka commented 3 years ago

Self hosted media server like Plex.tv, Emby.media and Jellyfin.org to utilize GPU to serve transcoded adaptive bitrate video streams in Dash/HLS to the PC/mobile client app.

Since not all client/display supports HDR and HEVC playback, the HDR-to-SDR tonemapping have to be done in the transcoding pipeline on the server side. Using a custom OpenCL kernel is feasible, but it way more convenient to get it done inside the AMFConverter along with the scaler.

Tonemapper like BT.2390, Hable or Reinhard are preferred.

In this case, for encoding to H.264 videos, AMFConverter also need to implement SURFACE_P010 to SURFACE_NV12 conversion.

For reference: HDR-to-SDR on Linux VAAPI(custom shaders built-in intel SDK only) https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_tonemap_vaapi.c