Issue with HDR Conversion using --vpp-ngx-truehdr Option

quamt commented 3 months ago

Hello,

I am experiencing an issue with the --vpp-ngx-truehdr option while attempting to perform AI-enhanced SDR to HDR conversion using RTX Video SDK with my NVIDIA GeForce RTX 3080. Despite following the recommended setup, I am not achieving the expected HDR output.

Here are the details of my setup and the commands I am using:

GPU: NVIDIA GeForce RTX 3080 (8704 cores, 1725 MHz) [PCIe4x16] Driver Version: 556.12 Operating System: Windows NVENC: 7.59

--codec hevc
--output-depth 10
--vpp-ngx-truehdr maxluminance=1200
--colormatrix bt2020nc 
--colorprim bt2020 
--transfer smpte2084

Despite this configuration, the output video is not in HDR as expected. Please advise if I am missing anything or if there are any additional steps I should take to achieve HDR output.

I would appreciate any guidance or suggestions regarding resolving this issue.

Thank you so much for your help.

rigaya commented 3 months ago

I am not achieving the expected HDR output.

Would you please let me know what exactly will be the meaning of "not achieving the expected HDR output"?

I was able to get HDR results with the options you've shown.

Encoded result playback with madVR HDR2SDR enabled (as I don't have HDR monitors)

Encoded result playback in SDR view (got dark as the encoded result is HDR)

quamt commented 3 months ago

Maybe it's a misunderstanding on my part. When I use MediaInfo, should it show HDR10 information? Ill try another encode.

quamt commented 3 months ago

I did run a test encode:

NVEncC (x64) 7.59 (r2957) by rigaya, Aug  2 2024 12:45:34 (VC 1929/Win)
OS Version     Windows 11 x64 (22631) [UTF-8]
CPU            12th Gen Intel Core i9-12900K [5.10GHz] (8P+8E,16C/24T)
GPU            #0: NVIDIA GeForce RTX 3080 (8704 cores, 1725 MHz)[PCIe4x16][560.70]
NVENC / CUDA   NVENC API 12.2, CUDA 12.6, schedule mode: auto
Input Buffers  CUDA, 46 frames
Input Info     avcuvid: H.264/AVC, 1920x1080, 24000/1001 fps
Vpp Filters    cspconv(nv12 -> yv12(16bit))
               deband: mode 1, range 15, threY 15, threCb 15, threCr 15
                       ditherY 15, ditherC 15, blurFirst no, randEachFrame no
               ngx-truehdr: colorspace: cspconv(yv12(16bit) -> yuv444(16bit))
                           matrix:bt709->GBR
                            cspconv(rgb(fp32) -> rgb32)
                            ngx-truehdr: contrast: 100, saturation: 100, middleGray: 50, maxLuminance: 1200
                            cspconv(rgba(fp16) -> rgb(fp32))
                            colorspace: prim:bt709->bt2020
                           transfer:linear->smpte2084
                           matrix:GBR->bt2020nc
               cspconv(yuv444(16bit) -> p010)
Output Info    H.265/HEVC main10 @ Level auto
               1920x1080p 1:1 23.976fps (24000/1001fps)
               avwriter: hevc, aac, subtitle#1 => matroska
Encoder Preset quality
Rate Control   CQP  I:19  P:21  B:23
ChromaQPOffset cb:0  cr:0
Split Enc Mode auto
Lookahead      on, 32 frames, Level 3, Adaptive I, B Insert
GOP length     240 frames
B frames       5 frames [ref mode: each]
Ref frames     12 frames, MultiRef L0:7 L1:3
AQ             off
CU max / min   auto / auto
VUI            matrix:bt2020nc,colorprim:bt2020,transfer:smpte2084,chromaloc:left
Others         mv:Q-pel repeat-headers

encoded 34286 frames, 67.79 fps, 1469.49 kbps, 250.50 MB
encode time 0:08:25, CPU: 0.3%, GPU: 64.4%, VE: 93.9%, VD: 10.4%, GPUClock: 1682MHz, VEClock: 1664MHz
frame type IDR   143
frame type I     143,  avgQP  19.00,  total size   17.27 MB
frame type P    5715,  avgQP  21.00,  total size  168.87 MB
frame type B   28428,  avgQP  23.00,  total size   64.36 MB

When checking the file with MediaInfo:

Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 23 min 50 s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 0)
Bit depth                                : 10 bits
Language                                 : Japanese
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant

I tried MPC-HC with MadVR, and it switched to HDR mode. Maybe I am overthinking the HDR info?

rigaya commented 3 months ago

When I use MediaInfo, should it show HDR10 information?

Unfortunately not. Please think that --vpp-ngx-truehdr just converts pixel data as a image filter, just like other vpp filters. Therefore, it does nothing about metadata.

You might try adding --mastering-display (with maxluminance you set) as HDR metadata if you need metadata.

quamt commented 3 months ago

You might try adding --mastering-display (with maxluminance you set) as HDR metadata if you need metadata.

Ok. If I understand that correctly, adding --master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(12000000,1)

Green Primary (G): G(13250,34500)
Blue Primary (B): B(7500,3000)
Red Primary (R): R(34000,16000)
White Point (WP): WP(15635,16450)
Luminance (L): L(12000000,1)

This should work, as all values need to be present, right? Are there any expected side effects? Thank you for your help.

ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 2 min 10 s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Language                                 : Japanese
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0001 cd/m2, max: 1200 cd/m2

quamt commented 3 months ago

Is the -vpp-ngx-truehdr only working with the 64-bit version of NVEnc? I'm asking because in the 32-bit version, I get the message: Error: Unknown option: --vpp-ngx-truehdr

rigaya commented 3 months ago

You might try adding --mastering-display (with maxluminance you set) as HDR metadata if you need metadata.

Ok. If I understand that correctly, adding --master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(12000000,1)

Yes, I think that shall be fine.

Is the -vpp-ngx-truehdr only working with the 64-bit version of NVEnc?

--vpp-ngx-truehdr only works for 64bit Windows version as RTX Video SDK only support 64bit.

Furthermore, CUDA was phasing out 32-bit support, and 32-bit builds are finally no longer available starting from CUDA 12.0, released in January 2023. (Therefore, NVEncC is build using CUDA 10.1).

32-bit version of NVEnc is maintained only for people with limited conditions, like someone using 32bit OS or someone who needs Avisynth 32bit plugins when using avs reader.

For others, I don't think there is need for 32bit version. Therefore, it is left with limited features, and there are no plans to enhance the 32bit version.

Please consider using 64bit version of NVEnc whenever possible. (This applies also for QSVEnc and VCEEnc)

quamt commented 3 months ago

Thank you very much for your help.

I will be closing this as all questions have been answered.

rigaya / NVEnc

Issue with HDR Conversion using --vpp-ngx-truehdr Option #610