mdhiggins / sickbeard_mp4_automator

Automatically convert video files to a standardized format with metadata tagging to create a beautiful and uniform media library
MIT License
1.52k stars 202 forks source link

How are we supposed to use the HDR part ? #1366

Closed Bigsausage64 closed 3 years ago

Bigsausage64 commented 3 years ago

Hi,

First let me congratulate you for your script, I've been using it for at least 1 year and it works flawlessly !!!!!

I have a Windows 10 machine, Python 3.8 64 bit, the latest version of your script, the latest build for FFMPEG and my standard use case works without any issue : converting all my files to MP4 using HEVCQSV (and all the audio and sub that I like...).

Now that Plex supports tone mapping I wanted to use the new HDR feature of your script to keep HDR metadata during transcoding.

Here is my Autoprocess.ini (I just copied the HDR configuration from the readme) : [Converter] ffmpeg = C:\Script\ffmpeg\ffmpegHDR.exe ffprobe = C:\Script\ffmpeg\ffprobeHDR.exe threads = 0 hwaccels = dxva2, cuvid, qsv, d3d11va, vaapi hwaccel-decoders = h264_cuvid, mjpeg_cuvid, mpeg1_cuvid, mpeg2_cuvid, mpeg4_cuvid, vc1_cuvid, hevc_qsv, h264_qsv, hevc_vaapi, h264_vaapi hwdevices = vaapi:/dev/dri/renderD128 hwaccel-output-format = vaapi:vaapi output-directory = output-format = mp4 output-extension = mp4 temp-extension = minimum-size = 0 ignored-extensions = copy-to = move-to = delete-original = False sort-streams = False process-same-extensions = False force-convert = False post-process = False wait-post-process = False detailed-progress = False preopts = -ss,00:10:00,-t,00:00:30 postopts =

[Permissions] chmod = 0777 uid = -1 gid = -1

[Metadata] relocate-moov = True full-path-guess = True tag = True tag-language = fre download-artwork = poster sanitize-disposition =

[Video] codec = hevcqsv max-bitrate = 50000 crf = -1 crf-profiles = preset = codec-parameters = dynamic-parameters = False max-width = 0 profile = main10 max-level = 0.0 pix-fmt = p010le filter = force-filter = False

[HDR] pix-fmt = space = bt2020nc transfer = smpte2084 primaries = bt2020 preset = codec-parameters = filter = force-filter = False

[Audio] codec = ac3 languages = eng,fr,fre,fra,spa,zh default-language = first-stream-of-language = True allow-language-relax = True channel-bitrate = 256 max-bitrate = 0 max-channels = 0 prefer-more-channels = True default-more-channels = False filter = force-filter = False sample-rates = copy-original = True aac-adtstoasc = False ignore-truehd = mp4, m4v ignored-dispositions = unique-dispositions = False

[Universal Audio] codec = aac channel-bitrate = 256 first-stream-only = False move-after = False filter = force-filter = False

[Subtitle] codec = ssrt codec-image-based = languages = eng default-language = eng first-stream-of-language = False encoding = UTF-8 burn-subtitles = False burn-dispositions = download-subs = True download-hearing-impaired-subs = False download-providers = opensubtitles,addic7ed,podnapisi,thesubdb embed-subs = False embed-image-subs = False embed-only-internal-subs = False filename-dispositions = forced ignore-embedded-subs = False ignored-dispositions = unique-dispositions = False attachment-codec =

When I test with Manual.py, a HDR stream is detected: 2020-11-26 16:11:26 - MANUAL - INFO - HDR video stream detected for 0.

But the resulting output has no HDR Metadata. We can definitely see that the generated command line is missing color space, range and trc info which I think should be embedded in my file so my LG TV recognize the file as an HDR one:

Here is the FFMPEG command line that is generated : C:\Script\ffmpeg\ffmpegHDR.exe -hwaccel dxva2 -ss 00:10:00 -t 00:00:30 -i test.mkv -vcodec hevc_qsv -map 0:0 -pix_fmt p010le -vb 4363k -metadata:s:v BPS=4363000 -metadata:s:v BPS-eng=4363000 -metadata:s:v title=4K HDR -metadata:s:v handler_name=4K HDR -profile:v main10 -tag:v hvc1 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=rus -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 copy -map 0:1 -metadata:s:a:1 title=5.1 Channel -metadata:s:a:1 handler_name=5.1 Channel -metadata:s:a:1 language=rus -disposition:a:1 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y C:\Script\Test\test.mp4

Here are the Mediainfo for Input : Codec HEVC Débit 4591 kbps Bit Depth 10 Chroma Subsampling 4:2:0 Coded Height 1608 Coded Width 3840 Color Primaries bt2020 Color Range tv Color Space bt2020nc Color Trc smpte2084 Frame Rate 23.976 fps Height 1608 Niveau 5.0 Profil main 10 Ref Frames 1 Width 3840 Display Title 4K (HEVC Main 10 HDR) Extended Display Title 4K (HEVC Main

Here are the Mediainfo for Output : Codec HEVC Débit 2931 kbps Bit Depth 10 Chroma Subsampling 4:2:0 Codec ID hvc1 Coded Height 1616 Coded Width 3840 Color Range tv Frame Rate 23.976 fps Height 1608 Niveau 5.0 Profil main 10 Ref Frames 1 Scan Type progressive Stream Identifier 1 Width 3840 Display Title 4K (HEVC Main 10) Extended Display Title 4K (HEVC Main 10)

Can you please help me ? HW transcoding is a prerequisite for me (I accept all the downsides). Let me know if I need to use some other codec, encoder, decoder or whatever (sorry I'm a bit of a noob)

I've been looking for some help on Internet but can't even figure out if what I want to achieve is feasible.

mdhiggins commented 3 years ago

Some HW transcoding will not write HDR metadata so be aware of that (NVidia for example does not). As far as the metadata is concerned, you need to either specify the metadata in the codec-parameters under the HDR and Video sections or turn on dynamic-parameters (which applies to both)

codec-parameters =
dynamic-parameters = False

This thread goes over a lot of the development and options related to HDR metadata

https://github.com/mdhiggins/sickbeard_mp4_automator/issues/1285

Bigsausage64 commented 3 years ago

Thanks for your support !!!!

Will try the dynamic parameters and see how it works.

Bigsausage64 commented 3 years ago

I usually use hevcqsv as my codec.

As it is HW, I understand HDR metadata won't be kept.

Will the dynamic parameters follow this pattern : when HDR is not detected in source, hevcqsv will be used by default. If HDR is detected in the source, codec is automatically switched to a SW one (like h265 for me) ?

If not, could you add this as a new feature ? fallback codec for example ?

Thanks again !!

mdhiggins commented 3 years ago

HEVC QSV might actually preserve metadata. I think Nvidia is the biggest culprit

Alternative codec for HDR would need to be developed

Sent from my iPhone

On Dec 4, 2020, at 2:38 PM, Bigsausage64 notifications@github.com wrote:

 I usually use hevcqsv as my codec.

As it is HW, I understand HDR metadata won't be kept.

Will the dynamic parameters follow this pattern : when HDR is not detected in source, hevcqsv will be used by default. If HDR is detected in the source, codec is automatically switched to a SW one (like h265 for me) ?

If not, could you add this as a new feature ? fallback codec for example ?

Thanks again !!

Le ven. 4 déc. 2020 à 20:33, Arnaud Delaunay arno.delaunay@gmail.com a écrit :

Thanks for your support !!!!

Will try the dynamic parameters and see how it works.

Le jeu. 26 nov. 2020 à 18:14, Michael Higgins notifications@github.com a écrit :

Some HW transcoding will not write HDR metadata so be aware of that (NVidia for example does not). As far as the metadata is concerned, you need to either specify the metadata in the codec-parameters under the HDR and Video sections or turn on dynamic-parameters (which applies to both)

codec-parameters = dynamic-parameters = False

This thread goes over a lot of the development and options related to HDR metadata

1285 https://github.com/mdhiggins/sickbeard_mp4_automator/issues/1285

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mdhiggins/sickbeard_mp4_automator/issues/1366#issuecomment-734411512, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKIYODWL2TNJVSNKERUTZNDSR2EGZANCNFSM4UD3O3WA .

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.

Bigsausage64 commented 3 years ago

Ok, I will try and let you know the outcome.

Bigsausage64 commented 3 years ago

Ok so I confirm it does not work with hevcqsv as a codec. And it does work when I use h265 as a codec

Media info of the source file : Vidéo ID : 1 Format : HEVC Format/Info : High Efficiency Video Coding Nom commercial : HDR10 Profil du format : Main 10@L5.1@High Identifiant du codec : V_MPEGH/ISO/HEVC Durée : 1 h 50 min Débit : 19,9 Mb/s Largeur : 3 840 pixels Hauteur : 2 160 pixels Format à l'écran : 16/9 Type d'images/s : Constant Images par seconde : 23,976 (24000/1001) Im/s Espace de couleurs : YUV Sous-échantillonnage de la chrominance : 4:2:0 (Type 2) Profondeur des couleurs : 10 bits Bits/(Pixel*Image) : 0.100 Taille du flux : 15,3 Gio (70%) Langue : Anglais Default : Oui Forced : Non Gamme de couleurs : Limited Coordonnées de chromaticité : BT.2020 Caractéristiques du transfert : PQ Coefficients de la matrice : BT.2020 non-constant MasteringDisplay_ColorPrimaries : Display P3 MasteringDisplay_Luminance : min: 0.0001 cd/m2, max: 1000 cd/m2

Media info of the HEVCQSV file : Vidéo ID : 1 Format : HEVC Format/Info : High Efficiency Video Coding Profil du format : Main 10@L5@High Identifiant du codec : hvc1 Identifiant du codec/Info : High Efficiency Video Coding Durée : 1 min 0s Débit : 21,3 Mb/s Largeur : 3 840 pixels Hauteur : 2 160 pixels Format à l'écran : 16/9 Type d'images/s : Constant Images par seconde : 23,976 (24000/1001) Im/s Norme : Component Espace de couleurs : YUV Sous-échantillonnage de la chrominance : 4:2:0 Profondeur des couleurs : 10 bits Type de balayage : Progressif Bits/(Pixel*Image) : 0.107 Taille du flux : 152 Mio (85%) Titre : 4K HDR / 4K HDR Langue : Anglais Gamme de couleurs : Limited Menus : 10

Media info of the H265 file : Vidéo ID : 1 Format : HEVC Format/Info : High Efficiency Video Coding Nom commercial : HDR10 Profil du format : Main 10@L5@High Identifiant du codec : hvc1 Identifiant du codec/Info : High Efficiency Video Coding Durée : 1 min 0s Débit : 32,0 Mb/s Largeur : 3 840 pixels Hauteur : 2 160 pixels Format à l'écran : 16/9 Type d'images/s : Constant Images par seconde : 23,976 (24000/1001) Im/s Espace de couleurs : YUV Sous-échantillonnage de la chrominance : 4:2:0 Profondeur des couleurs : 10 bits Type de balayage : Progressif Bits/(Pixel*Image) : 0.161 Taille du flux : 229 Mio (90%) Titre : 4K HDR / 4K HDR Bibliothèque utilisée : x265 3.2+35-acc5d16999e2:[Windows][GCC 9.2.1][64 bit] 10bit Paramètres d'encodage : cpuid=1111039 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 / scenecut=40 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=1 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / bitrate=28356 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=0 / display-window=0 / master-display=G(34000,16000)B(7500,3000)R(13250,34500)WP(15635,16450)L(1,10000000) / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.01 / no-opt-cu-delta-qp / no-aq-motion / hdr10 / hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / no-scenecut-aware-qpconformance-window-offsets / right=0 / bottom=0 Langue : Anglais Gamme de couleurs : Limited Coordonnées de chromaticité : BT.2020 Caractéristiques du transfert : PQ Coefficients de la matrice : BT.2020 non-constant Menus : 10 MasteringDisplay_ColorPrimaries : Display P3 MasteringDisplay_Luminance : min: 1000.0000 cd/m2, max: 0 cd/m2

For reference, my autoprocess.ini : [Converter] ffmpeg = C:\Script\ffmpeg\ffmpegHDR.exe ffprobe = C:\Script\ffmpeg\ffprobeHDR.exe threads = 0 hwaccels = dxva2, cuvid, qsv, d3d11va, vaapi hwaccel-decoders = h264_cuvid, mjpeg_cuvid, mpeg1_cuvid, mpeg2_cuvid, mpeg4_cuvid, vc1_cuvid, hevc_qsv, h264_qsv, hevc_vaapi, h264_vaapi hwdevices = vaapi:/dev/dri/renderD128 hwaccel-output-format = vaapi:vaapi output-directory = output-format = mp4 output-extension = mp4 temp-extension = minimum-size = 0 ignored-extensions = nfo, ds_store copy-to = move-to = delete-original = False sort-streams = True process-same-extensions = False force-convert = False post-process = False wait-post-process = False detailed-progress = False preopts = -ss,00:10:00,-t,00:01:00 postopts = -max_muxing_queue_size, 9999 bypass-if-copying-all = False

[Permissions] chmod = 0644 uid = -1 gid = -1

[Metadata] relocate-moov = True full-path-guess = True tag = True tag-language = fre download-artwork = poster sanitize-disposition = True

[Video] codec = The only setting that was changed between the 2 tests : hevcqsv then h265 max-bitrate = 50000 crf = -1 crf-profiles = preset = codec-parameters = dynamic-parameters = True max-width = 0 profile = main10 max-level = 0.0 pix-fmt = yuv420p10le filter = force-filter = False

[HDR] pix-fmt = yuv420p10le space = bt2020nc transfer = smpte2084 primaries = bt2020 preset = codec-parameters = filter = force-filter = False

[Audio] codec = ac3 languages = eng,fra,spa,zho default-language = first-stream-of-language = False allow-language-relax = True channel-bitrate = 256 max-bitrate = 0 max-channels = 0 prefer-more-channels = True default-more-channels = True filter = force-filter = False sample-rates = copy-original = True aac-adtstoasc = False ignore-truehd = mp4, m4v ignored-dispositions = unique-dispositions = False

[Universal Audio] codec = aac channel-bitrate = 256 first-stream-only = False move-after = False filter = force-filter = False

[Subtitle] codec = srt codec-image-based = languages = eng default-language = first-stream-of-language = False encoding = burn-subtitles = False burn-dispositions = download-subs = False download-hearing-impaired-subs = False download-providers = embed-subs = False embed-image-subs = False embed-only-internal-subs = False filename-dispositions = ignore-embedded-subs = False ignored-dispositions = unique-dispositions = False attachment-codec =

You said it should work, so am I doing something wrong ? By the way, I kept the "-max_muxing_queue_size, 9999" postop (when reading the thread, I was not sure if it was necessary or not........)

Thanks

Bigsausage64 commented 3 years ago

The HEVCQSV command line that was generated :

C:\Script\ffmpeg\ffmpegHDR.exe -hwaccel dxva2 -ss 00:10:00 -t 00:01:00 -i
C:\Script\HDR\Mission.Impossible.1996.Multi.2160p.UHD.BluRay.x265.HDR.TrueHD.5.1-DTOne.mkv
-vcodec hevc_qsv -map 0:0 -pix_fmt yuv420p10le -vb 28356k -metadata:s:v
BPS=28356000 -metadata:s:v BPS-eng=28356000 -metadata:s:v title=4K HDR
-metadata:s:v handler_name=4K HDR -profile:v main10 -x265-params
hdr-opt=1:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(34000,16000)B(7500,3000)R(13250,34500)WP(15635,16450)L(1,10000000):max-cll=0,0
-tag:v hvc1 -c:a:0 aac -map 0:2 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0
BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo
-metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng
-disposition:a:0
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-strict experimental -c:a:1 copy -map 0:2 -metadata:s:a:1 title=5.1 Channel
-metadata:s:a:1 handler_name=5.1 Channel -metadata:s:a:1 language=eng
-disposition:a:1
+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-c:a:2 aac -map 0:7 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000
-metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2
handler_name=Stereo -metadata:s:a:2 language=fra -disposition:a:2
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-strict experimental -c:a:3 copy -map 0:7 -metadata:s:a:3 title=5.1 Channel
-metadata:s:a:3 handler_name=5.1 Channel -metadata:s:a:3 language=fra
-disposition:a:3
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-c:a:4 aac -map 0:5 -ac:a:4 2 -b:a:4 256k -metadata:s:a:4 BPS=256000
-metadata:s:a:4 BPS-eng=256000 -metadata:s:a:4 title=Stereo -metadata:s:a:4
handler_name=Stereo -metadata:s:a:4 language=spa -disposition:a:4
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-strict experimental -c:a:5 copy -map 0:5 -metadata:s:a:5 title=5.1 Channel
-metadata:s:a:5 handler_name=5.1 Channel -metadata:s:a:5 language=spa
-disposition:a:5
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-c:a:6 aac -map 0:6 -ac:a:6 2 -b:a:6 256k -metadata:s:a:6 BPS=256000
-metadata:s:a:6 BPS-eng=256000 -metadata:s:a:6 title=Stereo -metadata:s:a:6
handler_name=Stereo -metadata:s:a:6 language=spa -disposition:a:6
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-strict experimental -c:a:7 copy -map 0:6 -metadata:s:a:7 title=5.1 Channel
-metadata:s:a:7 handler_name=5.1 Channel -metadata:s:a:7 language=spa
-disposition:a:7
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-f mp4 -threads 0 -metadata:g encoding_tool=SMA -max_muxing_queue_size 9999
-y
C:\Script\HDR\Mission.Impossible.1996.Multi.2160p.UHD.BluRay.x265.HDR.TrueHD.5.1-DTOne.mp4

The H265 command line that was generated :

C:\Script\ffmpeg\ffmpegHDR.exe -hwaccel dxva2 -ss 00:10:00 -t 00:01:00 -i
C:\Script\HDR\Mission.Impossible.1996.Multi.2160p.UHD.BluRay.x265.HDR.TrueHD.5.1-DTOne.mkv
-vcodec libx265 -map 0:0 -pix_fmt yuv420p10le -vb 28356k -metadata:s:v
BPS=28356000 -metadata:s:v BPS-eng=28356000 -metadata:s:v title=4K HDR
-metadata:s:v handler_name=4K HDR -profile:v main10 -x265-params
hdr-opt=1:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(34000,16000)B(7500,3000)R(13250,34500)WP(15635,16450)L(1,10000000):max-cll=0,0
-tag:v hvc1 -c:a:0 aac -map 0:2 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0
BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo
-metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng
-disposition:a:0
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-strict experimental -c:a:1 copy -map 0:2 -metadata:s:a:1 title=5.1 Channel
-metadata:s:a:1 handler_name=5.1 Channel -metadata:s:a:1 language=eng
-disposition:a:1
+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-c:a:2 aac -map 0:7 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000
-metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2
handler_name=Stereo -metadata:s:a:2 language=fra -disposition:a:2
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-strict experimental -c:a:3 copy -map 0:7 -metadata:s:a:3 title=5.1 Channel
-metadata:s:a:3 handler_name=5.1 Channel -metadata:s:a:3 language=fra
-disposition:a:3
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-c:a:4 aac -map 0:5 -ac:a:4 2 -b:a:4 256k -metadata:s:a:4 BPS=256000
-metadata:s:a:4 BPS-eng=256000 -metadata:s:a:4 title=Stereo -metadata:s:a:4
handler_name=Stereo -metadata:s:a:4 language=spa -disposition:a:4
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-strict experimental -c:a:5 copy -map 0:5 -metadata:s:a:5 title=5.1 Channel
-metadata:s:a:5 handler_name=5.1 Channel -metadata:s:a:5 language=spa
-disposition:a:5
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-c:a:6 aac -map 0:6 -ac:a:6 2 -b:a:6 256k -metadata:s:a:6 BPS=256000
-metadata:s:a:6 BPS-eng=256000 -metadata:s:a:6 title=Stereo -metadata:s:a:6
handler_name=Stereo -metadata:s:a:6 language=spa -disposition:a:6
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-strict experimental -c:a:7 copy -map 0:6 -metadata:s:a:7 title=5.1 Channel
-metadata:s:a:7 handler_name=5.1 Channel -metadata:s:a:7 language=spa
-disposition:a:7
-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions
-f mp4 -threads 0 -metadata:g encoding_tool=SMA -max_muxing_queue_size 9999
-y
C:\Script\HDR\Mission.Impossible.1996.Multi.2160p.UHD.BluRay.x265.HDR.TrueHD.5.1-DTOne.mp4

Something else that is weird in the h265 media info : MasteringDisplay_Luminance : min: 1000.0000 cd/m2, max: 0 cd/m2

Should that not be the opposite ? Min = 0 and Max 1000cd/m2 ? At least in the source it's in the "proper" order.

Bigsausage64 commented 3 years ago

Finally I checked the movie generated with h265.

HDR is properly triggered via the NVIDIA Shield on my tv and the movie looks good.

Once transcoded by PLEX (the latest version with HDR tone mapping) to my smartphone, we can see lot of defaults in the image (see capture attached), especially on the fingers.

Is this the result of the "weird" luminance metadata ?

Hope this helps

mdhiggins commented 3 years ago

bf8eb45bd6125d66231fca044253d26d5ee0f86b

Looks like I had them backwards, max comes before min

See if that fixes things

mdhiggins commented 3 years ago

bc934fd6c28244d63592edfd5ff9f80757648da6

That should add an extra codec option for HDR videos, will fall back to the old way if none are specified

Bigsausage64 commented 3 years ago

Just WOW !! You're fast !!!

I'm testing the new luminance setting. Will let you know the outcome but CPU encoding is so slowwwwwww

How do I setup my autoprocess.ini to use the https://github.com/mdhiggins/sickbeard_mp4_automator/commit/bc934fd6c28244d63592edfd5ff9f80757648da6 feature ?

Thanks a lot for your work !!! much appreciated

mdhiggins commented 3 years ago

When you update (and run the script once so it can update your config file) there will be a new codec parameter in the config under the HDR section that works the same as the codec parameter under video but will only be applied for HDR sources

Bigsausage64 commented 3 years ago

Got it. Will test and give a feedback. Thanks !

Bigsausage64 commented 3 years ago

I confirm, with https://github.com/mdhiggins/sickbeard_mp4_automator/commit/bf8eb45bd6125d66231fca044253d26d5ee0f86b, luminance issue is fixed while movie is transcoded by PLEX with HDR tone mapping activated 👍

Bigsausage64 commented 3 years ago

I also confirm https://github.com/mdhiggins/sickbeard_mp4_automator/commit/bc934fd6c28244d63592edfd5ff9f80757648da6 allows another codec to be used when HDR is detected !!!! fantastic, thank you

mdhiggins commented 3 years ago

Excellent, good catch on the luminance issue and glad the alt codec is working

and just to confirm with the new changes the HDR metadata still doesn't carry over with QSV? It does look like maybe VAAPI supports it and NVENC doesn't but can't find much on QSV. I don't know if maybe there is away to push some of the data with -bsf:v but couldn't find much info

Bigsausage64 commented 3 years ago

So I did a new test with:

[Video]
codec = hevcqsv
max-bitrate = 50000
crf = -1
crf-profiles = 
preset = 
codec-parameters = 
dynamic-parameters = True
max-width = 0
profile = main10
max-level = 0.0
pix-fmt = yuv420p10le
filter = 
force-filter = False

[HDR]
pix-fmt = yuv420p10le
space = bt2020nc
transfer = smpte2084
primaries = bt2020
preset = 
codec-parameters = 
filter = 
force-filter = False
codec = 

And I confirm metadata are not carried over:

Vidéo
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Profil du format                         : Main 10@L5@High
Identifiant du codec                     : hvc1
Identifiant du codec/Info                : High Efficiency Video Coding
Durée                                    : 1 min 0s
Débit                                    : 21,3 Mb/s
Largeur                                  : 3 840 pixels
Hauteur                                  : 2 160 pixels
Format à l'écran                         : 16/9
Type d'images/s                          : Constant
Images par seconde                       : 23,976 (24000/1001) Im/s
Norme                                    : Component
Espace de couleurs                       : YUV
Sous-échantillonnage de la chrominance   : 4:2:0
Profondeur des couleurs                  : 10 bits
Type de balayage                         : Progressif
Bits/(Pixel*Image)                       : 0.107
Taille du flux                           : 152 Mio (85%)
Titre                                    : 4K HDR / 4K HDR
Langue                                   : Anglais
Gamme de couleurs                        : Limited
Menus                                    : 10
Bigsausage64 commented 3 years ago

Regarding QSV and VAAPI, same as you, I kind of read it could be posible, but I've never found a ffmpeg option that would work...

I'd really like to have it work so if I can test things for you, I'd be glad to help

mdhiggins commented 3 years ago

I got nothing, I think this will likely require an FFMPEG update but at least the base codecs are working

Bigsausage64 commented 3 years ago

Ok, I'll be fine with the alt-codec solution you just made. Waiting for FFMPEG guys to do something about this

Bigsausage64 commented 3 years ago

Hi !

Is this what I'm waiting for ? https://patchwork.ffmpeg.org/project/ffmpeg/patch/20201202131826.10558-1-omondifredrick@gmail.com/

Do I just need to download the ffmpeg specific devel source that has the fix ?

Does this require some work on your script also ?

Thanks !!!

mdhiggins commented 3 years ago

I've seen a few patches trying to solve this and to apply that you'll need to download the FFMPEG source, apply the patch to modify the source code, and then compile that into an FFMPEG executable / binary.

Not sure if the script will need to be modified depends on how they implemented things.

mdhiggins commented 3 years ago

f48e2238ae27a122f45c54cac1e7df4057eaa763 be5f8c587df7714d6e67f62a11a67c00f11d7327

Looks like it just replaces the -x265-params argument with -qsv_params so I went ahead and made that change on a custom variant of the hevcqsv option

If you set your codec in autoProcess.ini to hevcqsvpatched that should be compatible with the patch. If that patch makes its way into the main FFMPEG builds I'll merge those changes into the main codecs but for now on non-patched FFMPEG is causes it to fail so keeping it separate

Bigsausage64 commented 3 years ago

Marvellous. I'll try tonight and will let you know.

I know nothing about how to compile a ffmpeg on windows, so I've downloaded the 64bit GPL vulkan shared release from this link: https://github.com/BtbN/FFmpeg-Builds/releases

It was compiled last night but don't know how to determine if patches are included or not......I guess I'll figure out when testing.......

I will keep you posted.

mdhiggins commented 3 years ago

Patch definitely won't be included in that, that patch you linked was just someone proposing that change, it has not been incorporated into FFMPEG yet

https://trac.ffmpeg.org/wiki/CompilationGuide

Bigsausage64 commented 3 years ago

Thanks for letting me know and not waste my time. Looks like I'm gonna learn something new tonight 😃

mdhiggins commented 3 years ago

You'll also need to apply the patch as a diff file

https://docs.cs-cart.com/latest/upgrade/apply_diff_file.html

Direct link to the diff file for that patch https://patchwork.ffmpeg.org/project/ffmpeg/patch/20201202131826.10558-1-omondifredrick@gmail.com/raw/

Bigsausage64 commented 3 years ago

Ok, long story short:

The patch needed a patch, there was a missing ")" in qsvenc.h, I added it manually and then ffmpeg got compiled without error and with every prerequisites hopefully:

ffmpeg version N-100446-geffea1a07f-g2749e20d04+2 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
  configuration:  --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-libfdk-aac --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-librav1e --enable-libsrt --enable-libgsm --enable-libvmaf --enable-libsvtav1 --enable-openssl --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads --enable-nonfree
  libavutil      56. 62.100 / 56. 62.100
  libavcodec     58.115.102 / 58.115.102
  libavformat    58. 65.100 / 58. 65.100
  libavdevice    58. 11.103 / 58. 11.103
  libavfilter     7. 93.100 /  7. 93.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100

When I launch a test with hevcqsvpatched codec, I believe the command line is generated as expected: C:\Script\ffmpeg\ffmpegPP1.exe -hwaccel dxva2 -ss 00:10:00 -t 00:01:00 -i \\DS3617XS\downloads\Radarr\input.mkv -vcodec hevc_qsv -map 0:0 -pix_fmt p010le -vb 30000k -metadata:s:v BPS=30000000 -metadata:s:v BPS-eng=30000000 -metadata:s:v title=4K HDR -metadata:s:v handler_name=4K HDR -profile:v main10 -qsv_params hdr-opt=1:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50):max-cll=1000,606 -tag:v hvc1 -c:a:0 aac -map 0:24 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 ac3 -map 0:24 -ac:a:1 6 -b:a:1 1536k -metadata:s:a:1 BPS=1536000 -metadata:s:a:1 BPS-eng=1536000 -metadata:s:a:1 title=7.1 Channel -metadata:s:a:1 handler_name=7.1 Channel -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:a:2 aac -map 0:25 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000 -metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2 handler_name=Stereo -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:3 copy -map 0:25 -metadata:s:a:3 title=5.1 Channel -metadata:s:a:3 handler_name=5.1 Channel -metadata:s:a:3 language=eng -disposition:a:3 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y \\DS3617XS\downloads\Radarr\output.mp4

But conversion fails:

[                    ] 0% Error converting file, FFMPEG error.
Traceback (most recent call last):
  File "C:\Script\PrePROD\resources\mediaprocessor.py", line 1554, in convert
    for timecode, debug in conv:
  File "C:\Script\PrePROD\converter\__init__.py", line 308, in convert
    for timecode, debug in self.ffmpeg.convert(outfile, optlist,
  File "C:\Script\PrePROD\converter\ffmpeg.py", line 715, in convert
    raise FFMpegConvertError('Exited with code %d' % p.returncode, cmd,
converter.ffmpeg.FFMpegConvertError: <unprintable FFMpegConvertError object>
[LOTS OF BLAH BLAH]
Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> hevc (hevc_qsv))
  Stream #0:24 -> #0:1 (truehd (native) -> aac (native))
  Stream #0:24 -> #0:2 (truehd (native) -> ac3 (native))
  Stream #0:25 -> #0:3 (ac3 (native) -> aac (native))
  Stream #0:25 -> #0:4 (copy)
Press [q] to stop, [?] for help
[hevc @ 000001d6ff247340] First slice in a frame missing.
    Last message repeated 6 times
[hevc @ 000001d6ff24ca00] First slice in a frame missing.
    Last message repeated 6 times
Invalid value for hdr-opt: 1.
[hevc_qsv @ 000001d6ff249c40] Invalid parameter name: invalid video parameters (-15)
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 000001d6ff24d780] Qavg: 187.258
[aac @ 000001d6ff24d780] 2 frames left in the queue on closing
[aac @ 000001d6ff247c40] Qavg: 204.071
[aac @ 000001d6ff247c40] 2 frames left in the queue on closing
Conversion failed!

I tested with different inputs, it's always the same error. Just for some other files, I also have this:

[matroska,webm @ 00000201254dc2c0] Could not find codec parameters for stream 4 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options

Here is my Autoprocess.ini:

[Converter]
ffmpeg = C:\Script\ffmpeg\ffmpegPP1.exe
ffprobe = C:\Script\ffmpeg\ffprobePP1.exe
threads = 0
hwaccels = dxva2, d3d11va, qsv, vaapi, cuvid
hwaccel-decoders = hevc_qsv, hevc_vaapi, h264_cuvid, mjpeg_cuvid, mpeg1_cuvid, mpeg2_cuvid, mpeg4_cuvid, vc1_cuvid, h264_qsv, h264_vaapi
hwdevices = vaapi:/dev/dri/renderD128
hwaccel-output-format = vaapi:vaapi
output-directory = 
output-format = mp4
output-extension = mp4
temp-extension = 
minimum-size = 0
ignored-extensions = nfo, ds_store
copy-to = 
move-to = 
delete-original = False
sort-streams = True
process-same-extensions = False
force-convert = False
post-process = False
wait-post-process = False
detailed-progress = True
preopts = -ss,00:10:00,-t,00:01:00
postopts = 
bypass-if-copying-all = False

[Video]
codec = hevcqsvpatched
max-bitrate = 30000
crf = -1
crf-profiles = 
preset = 
codec-parameters = 
dynamic-parameters = True
max-width = 0
profile = main10
max-level = 0.0
pix-fmt = p010le
filter = 
force-filter = False

[HDR]
pix-fmt = p010le
space = bt2020nc
transfer = smpte2084
primaries = bt2020
preset = 
codec-parameters = 
filter = 
force-filter = False
codec = 

Can you help ?

Bigsausage64 commented 3 years ago

@mdhiggins sorry for spamming, I cleaned my mess, please have a look

mdhiggins commented 3 years ago

Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options

Try adding -probesize, 50M, -analyzeduration, 100M to the preopts of autoProcess.ini and see if that helps

Bigsausage64 commented 3 years ago

Same result with preopts = -probesize,50000000,-analyzeduration,100000000

FFmpeg command:

C:\Script\ffmpeg\ffmpegPP.exe -hwaccel dxva2 -probesize 50000000 -analyzeduration 100000000 -ss 00:10:00 -t 00:01:00 -i \\DS3617XS\downloads\Radarr\input.mkv -vcodec hevc_qsv -map 0:0 -pix_fmt p010le -vb 30000k -metadata:s:v BPS=30000000 -metadata:s:v BPS-eng=30000000 -metadata:s:v title=4K HDR -metadata:s:v handler_name=4K HDR -profile:v main10 -qsv_params hdr-opt=1:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(40000000,50):max-cll=725,162 -tag:v hvc1 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 6 -b:a:1 1536k -metadata:s:a:1 BPS=1536000 -metadata:s:a:1 BPS-eng=1536000 -metadata:s:a:1 title=7.1 Channel -metadata:s:a:1 handler_name=7.1 Channel -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:a:2 aac -map 0:2 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000 -metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2 handler_name=Stereo -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:3 copy -map 0:2 -metadata:s:a:3 title=5.1 Channel -metadata:s:a:3 handler_name=5.1 Channel -metadata:s:a:3 language=eng -disposition:a:3 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y \\DS3617XS\downloads\Radarr\output.mp4
======================
[                    ] 0% Error converting file, FFMPEG error.
Traceback (most recent call last):
  File "C:\Script\PrePROD\resources\mediaprocessor.py", line 1554, in convert
    for timecode, debug in conv:
  File "C:\Script\PrePROD\converter\__init__.py", line 308, in convert
    for timecode, debug in self.ffmpeg.convert(outfile, optlist,
  File "C:\Script\PrePROD\converter\ffmpeg.py", line 715, in convert
    raise FFMpegConvertError('Exited with code %d' % p.returncode, cmd,
converter.ffmpeg.FFMpegConvertError: <unprintable FFMpegConvertError object>

Consider increasing the value for the 'analyzeduration' (100000000) and 'probesize' (50000000) options

Stream mapping:
  Stream #0:0 -> #0:0 (hevc (native) -> hevc (hevc_qsv))
  Stream #0:1 -> #0:1 (truehd (native) -> aac (native))
  Stream #0:1 -> #0:2 (truehd (native) -> ac3 (native))
  Stream #0:2 -> #0:3 (ac3 (native) -> aac (native))
  Stream #0:2 -> #0:4 (copy)
Press [q] to stop, [?] for help
Invalid value for hdr-opt: 1.e=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A
[hevc_qsv @ 000002075292bd40] Invalid parameter name: invalid video parameters (-15)
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0000020752928200] Qavg: 183.042
[aac @ 0000020752928200] 2 frames left in the queue on closing
[aac @ 00000207528e2080] Qavg: 176.515
[aac @ 00000207528e2080] 2 frames left in the queue on closing
Conversion failed!

I also tried with preopts = -probesize,100000000,-analyzeduration,900000000, same result.

Have you seen this line:

Invalid value for hdr-opt: 1.e=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A

It seems hdr-opt:1 is not accepted !

mdhiggins commented 3 years ago

Try taking some of those parameters out and see if it matters

C:\Script\ffmpeg\ffmpegPP.exe -hwaccel dxva2 -probesize 50000000 -analyzeduration 100000000 -ss 00:10:00 -t 00:01:00 -i \\DS3617XS\downloads\Radarr\input.mkv -vcodec hevc_qsv -map 0:0 -pix_fmt p010le -vb 30000k -metadata:s:v BPS=30000000 -metadata:s:v BPS-eng=30000000 -metadata:s:v title=4K HDR -metadata:s:v handler_name=4K HDR -profile:v main10 -qsv_params colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(40000000,50):max-cll=725,162 -tag:v hvc1 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 6 -b:a:1 1536k -metadata:s:a:1 BPS=1536000 -metadata:s:a:1 BPS-eng=1536000 -metadata:s:a:1 title=7.1 Channel -metadata:s:a:1 handler_name=7.1 Channel -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:a:2 aac -map 0:2 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000 -metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2 handler_name=Stereo -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:3 copy -map 0:2 -metadata:s:a:3 title=5.1 Channel -metadata:s:a:3 handler_name=5.1 Channel -metadata:s:a:3 language=eng -disposition:a:3 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y \\DS3617XS\downloads\Radarr\output.mp4

Removed the hdr-opt and repeat-headers parameters since looking at the code for the patch it seems these aren't supported

If that works I can filter them out on the script side of things

Could also push the probesize and duration further

-probesize, 500M, -analyzeduration, 1000M

Bigsausage64 commented 3 years ago

When using the command line above:

[NULL @ 00000295dce106c0] Unable to find a suitable output format for 'HDR'
HDR: Invalid argument
mdhiggins commented 3 years ago

Anything that has a space in it needs to wrapped in quotes for windows

C:\Script\ffmpeg\ffmpegPP.exe -hwaccel dxva2 -probesize 50000000 -analyzeduration 100000000 -ss 00:10:00 -t 00:01:00 -i \\DS3617XS\downloads\Radarr\input.mkv -vcodec hevc_qsv -map 0:0 -pix_fmt p010le -vb 30000k -metadata:s:v BPS=30000000 -metadata:s:v BPS-eng=30000000 -metadata:s:v title="4K HDR" -metadata:s:v handler_name="4K HDR" -profile:v main10 -qsv_params colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(40000000,50):max-cll=725,162 -tag:v hvc1 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 6 -b:a:1 1536k -metadata:s:a:1 BPS=1536000 -metadata:s:a:1 BPS-eng=1536000 -metadata:s:a:1 title="7.1 Channel" -metadata:s:a:1 handler_name="7.1 Channel" -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:a:2 aac -map 0:2 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000 -metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2 handler_name=Stereo -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:3 copy -map 0:2 -metadata:s:a:3 title="5.1 Channel" -metadata:s:a:3 handler_name="5.1 Channel" -metadata:s:a:3 language=eng -disposition:a:3 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y \\DS3617XS\downloads\Radarr\output.mp4

Try that, think I caught them all

Bigsausage64 commented 3 years ago
Invalid value for max-cll: 725,162.   0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A
[hevc_qsv @ 0000017569144500] Invalid value: invalid video parameters (-15)
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0000017569144e40] Qavg: 183.042
[aac @ 0000017569144e40] 2 frames left in the queue on closing
[aac @ 000001756a55c280] Qavg: 176.515
[aac @ 000001756a55c280] 2 frames left in the queue on closing
Bigsausage64 commented 3 years ago

Increasing -probesize to 500M and -analyzeduration to 1000M didn't change anything

mdhiggins commented 3 years ago

Looks like this patch set a min cll to 400 and a max to 1000 for some reason so your 725 / 162 is out of bounds for this source. You could manually change it to 400 just to see

C:\Script\ffmpeg\ffmpegPP.exe -hwaccel dxva2 -probesize 50000000 -analyzeduration 100000000 -ss 00:10:00 -t 00:01:00 -i \\DS3617XS\downloads\Radarr\input.mkv -vcodec hevc_qsv -map 0:0 -pix_fmt p010le -vb 30000k -metadata:s:v BPS=30000000 -metadata:s:v BPS-eng=30000000 -metadata:s:v title="4K HDR" -metadata:s:v handler_name="4K HDR" -profile:v main10 -qsv_params colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(40000000,50):max-cll=725,400 -tag:v hvc1 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 6 -b:a:1 1536k -metadata:s:a:1 BPS=1536000 -metadata:s:a:1 BPS-eng=1536000 -metadata:s:a:1 title="7.1 Channel" -metadata:s:a:1 handler_name="7.1 Channel" -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:a:2 aac -map 0:2 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000 -metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2 handler_name=Stereo -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:3 copy -map 0:2 -metadata:s:a:3 title="5.1 Channel" -metadata:s:a:3 handler_name="5.1 Channel" -metadata:s:a:3 language=eng -disposition:a:3 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y \\DS3617XS\downloads\Radarr\output.mp4
mdhiggins commented 3 years ago

Not sure why the patch designer set those values and would maybe try compiling with a larger range (0-2000?) by modifying the patch code and recompiling

Bigsausage64 commented 3 years ago

With min cll set manually to 400, it worked !!!!!!!!!!!!!!!!!!!!!!!!!!!

Will try the new compilation with a larger range of 0-2000..........

THANKS A LOT

mdhiggins commented 3 years ago
> +    const int maximum_cll = 1000;
> +    const int minimum_cll = 400;

Change those 2 values to 2000 and 0 respectively

mdhiggins commented 3 years ago

3a2c081d78f13d99cdb598cb8e7ab087c6d55ccd

That patch should filter out the hdr-opt and repeat-headers params when using this experimental patch, so script should work now

Bigsausage64 commented 3 years ago

You rock !!!!!!!!!!!

Bigsausage64 commented 3 years ago

After recompilation with:

  • const int maximum_cll = 2000;
  • const int minimum_cll = 0;

and https://github.com/mdhiggins/sickbeard_mp4_automator/commit/3a2c081d78f13d99cdb598cb8e7ab087c6d55ccd GPU encoding completes but the output doesn't get HDR metadata. I tested on 3 different movies

Input ffprobe test:

"frames": [
        {
            "pix_fmt": "yuv420p10le",
            "color_space": "bt2020nc",
            "color_primaries": "bt2020",
            "color_transfer": "smpte2084",
            "side_data_list": [
                {
                    "side_data_type": "Mastering display metadata",
                    "red_x": "34000/50000",
                    "red_y": "16000/50000",
                    "green_x": "13250/50000",
                    "green_y": "34500/50000",
                    "blue_x": "7500/50000",
                    "blue_y": "3000/50000",
                    "white_point_x": "15635/50000",
                    "white_point_y": "16450/50000",
                    "min_luminance": "1/10000",
                    "max_luminance": "10000000/10000"
                },
                {
                    "side_data_type": "Content light level metadata",
                    "max_content": 0,
                    "max_average": 0
                },
                {
                    "side_data_type": "H.26[45] User Data Unregistered SEI message"
                },
                {
                    "side_data_type": "H.26[45] User Data Unregistered SEI message"
                }
            ]
        }
    ]
}

output ffprobe test:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0000018a03ce1240] stream 0, timescale not set
Unsupported codec with id 100359 for input stream 6
    "frames": [
        {
            "pix_fmt": "yuvj420p",
            "color_space": "bt470bg",
            "side_data_list": [
                {
                    "side_data_type": "QP table data"
                },
                {
                    "side_data_type": "QP table properties"
                }
            ]
        }
    ]
}
Bigsausage64 commented 3 years ago

With the following manual command it works (GPU encoding + HDR Metadata are kept): C:\Git>C:\Script\ffmpeg\ffmpegPP.exe -hwaccel dxva2 -probesize 50000000 -analyzeduration 100000000 -ss 00:10:00 -t 00:01:00 -i \\DS3617XS\downloads\Radarr\input.mkv -vcodec hevc_qsv -map 0:0 -pix_fmt p010le -vb 30000k -metadata:s:v BPS=30000000 -metadata:s:v BPS-eng=30000000 -metadata:s:v title="4K HDR" -metadata:s:v handler_name="4K HDR" -profile:v main10 -qsv_params colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(40000000,50):max-cll=725,400 -tag:v hvc1 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 6 -b:a:1 1536k -metadata:s:a:1 BPS=1536000 -metadata:s:a:1 BPS-eng=1536000 -metadata:s:a:1 title="7.1 Channel" -metadata:s:a:1 handler_name="7.1 Channel" -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:a:2 aac -map 0:2 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000 -metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2 handler_name=Stereo -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:3 copy -map 0:2 -metadata:s:a:3 title="5.1 Channel" -metadata:s:a:3 handler_name="5.1 Channel" -metadata:s:a:3 language=eng -disposition:a:3 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y \\DS3617XS\downloads\Radarr\output.mp4

When using manual.py it encodes via GPU but looses the HDR Metadata. Here is the command generated: C:\Script\ffmpeg\ffmpegPP.exe -hwaccel dxva2 -probesize 500000000 -analyzeduration 1000000000 -ss 00:10:00 -t 00:01:00 -i \\DS3617XS\downloads\Radarr\input.mkv -vcodec hevc_qsv -map 0:0 -pix_fmt p010le -vb 30000k -metadata:s:v BPS=30000000 -metadata:s:v BPS-eng=30000000 -metadata:s:v title=4K HDR -metadata:s:v handler_name=4K HDR -profile:v main10 -qsv_params colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(40000000,50):max-cll=725,162 -tag:v hvc1 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 6 -b:a:1 1536k -metadata:s:a:1 BPS=1536000 -metadata:s:a:1 BPS-eng=1536000 -metadata:s:a:1 title=7.1 Channel -metadata:s:a:1 handler_name=7.1 Channel -metadata:s:a:1 language=eng -disposition:a:1 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -c:a:2 aac -map 0:2 -ac:a:2 2 -b:a:2 256k -metadata:s:a:2 BPS=256000 -metadata:s:a:2 BPS-eng=256000 -metadata:s:a:2 title=Stereo -metadata:s:a:2 handler_name=Stereo -metadata:s:a:2 language=eng -disposition:a:2 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:3 copy -map 0:2 -metadata:s:a:3 title=5.1 Channel -metadata:s:a:3 handler_name=5.1 Channel -metadata:s:a:3 language=eng -disposition:a:3 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y \\DS3617XS\downloads\Radarr\output.mp4

Bigsausage64 commented 3 years ago

The only differences are the quotes around titles and handler_name values !

Thanks 👍

mdhiggins commented 3 years ago

Can you post the ffprobe framedata for the file you're using in the command listed above? (the input.mkv file). It's calculating the max-cll to be 725,162 and I just want to make sure that's correct

Looks like that 400 is a hard cutoff for some reason though I'm not sure why

Bigsausage64 commented 3 years ago

Here it is:

"frames": [
        {
            "pix_fmt": "yuv420p10le",
            "color_space": "bt2020nc",
            "color_primaries": "bt2020",
            "color_transfer": "smpte2084",
            "side_data_list": [
                {
                    "side_data_type": "Mastering display metadata",
                    "red_x": "35400/50000",
                    "red_y": "14600/50000",
                    "green_x": "8500/50000",
                    "green_y": "39850/50000",
                    "blue_x": "6550/50000",
                    "blue_y": "2300/50000",
                    "white_point_x": "15635/50000",
                    "white_point_y": "16450/50000",
                    "min_luminance": "50/10000",
                    "max_luminance": "40000000/10000"
                },
                {
                    "side_data_type": "Content light level metadata",
                    "max_content": 725,
                    "max_average": 162
Bigsausage64 commented 3 years ago

another input file gives:

{
    "frames": [
        {
            "pix_fmt": "yuv420p10le",
            "color_space": "bt2020nc",
            "color_primaries": "bt2020",
            "color_transfer": "smpte2084",
            "side_data_list": [
                {
                    "side_data_type": "Mastering display metadata",
                    "red_x": "34000/50000",
                    "red_y": "16000/50000",
                    "green_x": "13250/50000",
                    "green_y": "34500/50000",
                    "blue_x": "7500/50000",
                    "blue_y": "3000/50000",
                    "white_point_x": "15635/50000",
                    "white_point_y": "16450/50000",
                    "min_luminance": "50/10000",
                    "max_luminance": "40000000/10000"
                },
                {
                    "side_data_type": "Content light level metadata",
                    "max_content": 1000,
                    "max_average": 606
                },
                {
                    "side_data_type": "HDR Dynamic Metadata SMPTE2094-40 (HDR10+)",
                    "application version": 1,
                    "num_windows": 1,
                    "targeted_system_display_maximum_luminance": "400/1",
                    "maxscl": "0/100000",
                    "maxscl": "0/100000",
                    "maxscl": "0/100000",
                    "average_maxrgb": "0/100000",
                    "num_distribution_maxrgb_percentiles": 9,
                    "distribution_maxrgb_percentage": 1,
                    "distribution_maxrgb_percentile": "0/100000",
                    "distribution_maxrgb_percentage": 5,
                    "distribution_maxrgb_percentile": "0/100000",
                    "distribution_maxrgb_percentage": 10,
                    "distribution_maxrgb_percentile": "100/100000",
                    "distribution_maxrgb_percentage": 25,
                    "distribution_maxrgb_percentile": "0/100000",
                    "distribution_maxrgb_percentage": 50,
                    "distribution_maxrgb_percentile": "0/100000",
                    "distribution_maxrgb_percentage": 75,
                    "distribution_maxrgb_percentile": "0/100000",
                    "distribution_maxrgb_percentage": 90,
                    "distribution_maxrgb_percentile": "0/100000",
                    "distribution_maxrgb_percentage": 95,
                    "distribution_maxrgb_percentile": "0/100000",
                    "distribution_maxrgb_percentage": 99,
                    "distribution_maxrgb_percentile": "0/100000",
                    "fraction_bright_pixels": "0/1000",
                    "knee_point_x": "0/4095",
                    "knee_point_y": "0/4095",
                    "num_bezier_curve_anchors": 9,
                    "bezier_curve_anchors": "102/1023",
                    "bezier_curve_anchors": "205/1023",
                    "bezier_curve_anchors": "307/1023",
                    "bezier_curve_anchors": "410/1023",
                    "bezier_curve_anchors": "512/1023",
                    "bezier_curve_anchors": "614/1023",
                    "bezier_curve_anchors": "717/1023",
                    "bezier_curve_anchors": "819/1023",
                    "bezier_curve_anchors": "922/1023"
                }
            ]
        }
    ]
}
Bigsausage64 commented 3 years ago

Other one:

"frames": [
        {
            "pix_fmt": "yuv420p10le",
            "color_space": "bt2020nc",
            "color_primaries": "bt2020",
            "color_transfer": "smpte2084",
            "side_data_list": [
                {
                    "side_data_type": "Mastering display metadata",
                    "red_x": "35400/50000",
                    "red_y": "14600/50000",
                    "green_x": "8500/50000",
                    "green_y": "39850/50000",
                    "blue_x": "6550/50000",
                    "blue_y": "2300/50000",
                    "white_point_x": "15635/50000",
                    "white_point_y": "16450/50000",
                    "min_luminance": "20/10000",
                    "max_luminance": "10000000/10000"
                },
                {
                    "side_data_type": "Content light level metadata",
                    "max_content": 992,
                    "max_average": 518
                }
            ]
        }
mdhiggins commented 3 years ago

Do do the files where the "max_average" is above 400 work fine?

Edit: work fine with the script specifically