search

rigaya / NVEnc

NVENCによる高速エンコードの性能実験
https://rigaya34589.blog.fc2.com/blog-category-17.html
Other
1.04k stars 108 forks source link

4.02 is the last version finishing 4K HDR encoding longer than 30 minutes. #56

Open techok opened 6 years ago

techok commented 6 years ago

Just wanted to give this important feedback. Using StaxRIP the last NvEncC version finishing long video 4K HDR encoding is the 4.02

Any other version always crashes at around 35% of encoding a 2 hours file. No matter the setup or hardware, it always crashes on the same point.

Also tried different files to encode so the files are not the issue but the NvEncC new versions.

rigaya commented 6 years ago

I tried encoding with command line below, but was unsuccessful to reproduce the issue. As I don't have long 4K movies, the input is 1920x1080, and resized to 4K.

x64\NVEncC64.exe -i "input.ts" -o "output.mp4" --output-res 3840x2160 -c hevc --profile main10 --max-cll 552,190 --master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(39999999,50)

Would you provide the exact command line and the log when the issue occurs?

techok commented 6 years ago

Hello Rigaya, here the command ( Using Latest StaxRip x64 ) :

Tried to encode a 1080p 2 hours upscaled video by StaxRip Vapoursynth Resize in Spline, plus some other vapoursynth filters like SMDegrain. This always fails in 4.03 and up. It finishes perfectly fine using 4.02.

NVEncC64 --sar %target_sar% --codec h265 --vbrhq 60000 --vbr-quality 12 --output-depth 10 --lookahead 32 --preset quality --aq --aq-strength 3 --ref 5 --output-buf 128 --lookahead 32 --qp-init 1 --gop-len 240 --max-bitrate 120000 --mv-precision q-pel --colorprim bt2020 --colormatrix bt2020nc --transfer smpte2084 --master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,50) --max-cll '1000,250' -i "%script_file%" -o "%encoder_out_file%"

lgoncal commented 6 years ago

Sorry to but in however I am using 4.07 presently and have no issues with long encodes (2hr +) with this or previous versions. I use vpy script as input This is my command line to which I had the info for hdr "C:\Program Files (Portable)\StaxRip\Apps\NVEnc\NVEncC64.exe" --vbrhq 38400 --codec h265 --preset quality --level 5.1 --output-depth 10 --ref 5 --gop-len 24 --lookahead 32 --qp-init 1 --max-bitrate 38400 --vbr-quality 25 --aq --cuda-schedule auto --mv-precision q-pel --cabac -i -o T:\TEMP\%source_name%_temp\_out.h265

Remember that StaxRip hasn't been updated for a while and it is still configured for version 3.xx

techok commented 6 years ago

Here it always fails and you are not using vapoursynth filters, SMDegrain is being used on the script. The crash is always at around 38% of a 2 hour long video upscaled to 4K and using SMDegrain. With 4.02 the encoding completes, with higher version it does not.

Not sure what do you mean by StaxRip not being updated as what you can do is just overwrite the NvEncC64 dll's to the /apps/NVEnc and it will be using the version you put on there instead.

lgoncal commented 6 years ago

Using DGSource, which is a Vapoursynth source filter. You can overwrite, but is Staxrip and its filters handling the new functions of NVEncC properly? Try without SMDegrain, or a different source filter, to make sure problem is not outside of NVEncC

Here it always fails and you are not using vapoursynth filters

Maybe the problem is the filter

techok commented 6 years ago

Yes probably but why it works with 4.02 and not with latest versions? SMDegrain must be used as its the most effective ( personal taste ) .

rigaya commented 6 years ago

I was able to run NVEncC properly with the same commadline using vpy (LWLibavSource) for 2 hours source, but without any filtering in vapoursynth.

input.vpy

#!/usr/bin/python3
import vapoursynth as vs
import sys
core = vs.get_core(threads=8)
clip = core.lsmas.LWLibavSource('<input.ts>')
clip = core.resize.Spline36(clip,width=3840,height=2160,format=vs.YUV420P10)
clip.set_output()

log

NVEncC (x64) 4.07 (r792) by rigaya, Jun 13 2018 20:40:11 (VC 1900/Win/avx2)
OS Version     Windows 10 x64 (17134)
CPU            Intel Core i9-7980XE @ 2.60GHz [TB: 4.21GHz] (18C/36T)
GPU            #0: GeForce GTX 1060 6GB (10 EU) @ 1708 MHz (397.64)
NVENC / CUDA   NVENC API 8.1, CUDA 9.2, schedule mode: auto
Input Buffers  CUDA, 41 frames
Input Info     VapourSynthMT r39(yv12(10bit))->p010 [AVX2], 3840x2160, 30000/1001 fps
Vpp Filters    copyHtoD
Output Info    H.265/HEVC main10 @ Level auto
               3840x2160p 1:1 29.970fps (30000/1001fps)
               avwriter: hevc => mp4
Encoder Preset quality
Rate Control   VBRHQ
Bitrate        60000 kbps (Max: 120000 kbps)
Target Quality 12.00
Initial QP     I:20  P:23  B:25
VBV buf size   auto
Lookahead      on, 32 frames, Adaptive I, B Insert
GOP length     240 frames
B frames       0 frames
Ref frames     5 frames, LTR: off
AQ             on
CU max / min   auto / auto
Others         mv:Q-pel

encoded 162460 frames, 69.13 fps, 60484.48 kbps, 39085.29 MB
encode time 0:39:10, CPU: 8.7%, GPU: 25.5%, VE: 100.0%, GPUClock: 1874MHz, VEClock: 1683MHz
frame type IDR    794
frame type I      794,  avgQP  14.81,  total size    452.59 MB
frame type P   161666,  avgQP  16.03,  total size  38632.70 MB

Actually it would be hard to know what is the problem if I can't reproduce the problem, so would you provide the vpy file and the log as above, as I'm not familiar in StaxRip or SMDegrain.

pwacooijmans commented 6 years ago

--aq strength is not supported for h265 if I am correct NVEncC (x64) 4.06 (r784) by rigaya, Jun 2 2018 18:10:09 (VC 1900/Win/avx2) [NVENC API v8.1, CUDA 8.0] reader: raw, avi, avs, vpy, avhw [H.264/AVC, H.265/HEVC, MPEG2, VP8, VP9, MPEG1, MPEG4] Usage: NVEncC.exe [Options] -i -o

Input can be avi, avs, raw YUV, YUV4MPEG2(y4m). When Input is in raw format, fps, input-res is required.

Ouput format will be in raw H.264/AVC or H.265/HEVC ES.

Example: NVEncC -i "" -o "" avs2pipemod -y4mp "" | NVEncC --y4m -i - -o ""

Information Options: -h,-? --help print help -v,--version print version info --check-device show DeviceId for GPUs available on system --check-hw [] check NVEnc codecs for specified DeviceId if unset, will check DeviceId #0 --check-features [] check for NVEnc Features for specified DeviceId if unset, will check DeviceId #0 --check-environment check for Environment Info --check-avversion show dll version --check-codecs show codecs available --check-encoders show audio encoders available --check-decoders show audio decoders available --check-formats show in/out formats available --check-protocols show in/out protocols available --check-filters show filters available

Basic Encoding Options: -d,--device set DeviceId used in NVEnc (default:-1 as auto) use --check-device to show device ids.

-i,--input set input filename -o,--output set output filename

Input formats (auto detected from extension of not set) --raw set input as raw format --y4m set input as y4m format --avi set input as avi format --avs set input as avs format --vpy set input as vpy format --vpy-mt set input as vpy(mt) format --avhw [] use libavformat + cuvid for input this enables full hw transcode and resize. avhw mode could be set as a option

-c,--codec set output codec h264 (or avc), h265 (or hevc) --profile set codec profile H.264: baseline, main, high(default), high444 HEVC : main, main10, main444 --tier set codec tier HEVC : main, high --lossless for lossless (YUV444 only) / default: off --level set codec level

pwacooijmans commented 6 years ago

NVEncC64 --master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1) --max-cll 1000,640 --sar %target_sar% --vbr 25000 --codec h265 --preset quality --output-depth 10 --lookahead 32 --output-buf 128 --cuda-schedule auto --no-b-adapt --no-i-adapt --videoformat pal --max-bitrate 40000 --vbv-bufsize 160000 --aq --colormatrix bt2020nc --colorprim bt2020 --transfer smpte2084 --mv-precision q-pel --chromaloc 2 -i "%script_file%" -o "%encoder_out_file%"

This is just an example....but works fine for me, with nvidia driver 397.93 and version 4.06 of nvenc

pwacooijmans commented 6 years ago

NVEncC64 --sar %target_sar% --vbr 25000 --codec h264 --preset quality --output-depth 8 --lookahead 32 --output-buf 128 --cuda-schedule auto --no-b-adapt --no-i-adapt --videoformat pal --max-bitrate 40000 --vbv-bufsize 50000 --aq-temporal --mv-precision q-pel -i "%script_file%" -o "%encoder_out_file%" In this case you can set an --aq-strength if you like.

pwacooijmans commented 6 years ago

--no-b-adapt disable adapt. B frame insertion for lookahead mode only, default: off Plus I am not sure this is required or not but might be an option you have to add for lookahead.

techok commented 6 years ago

Hello, update:

Using StaxRip internal Nvidia encoder option ( i was using the command line option ) it did not crash using the 4.04 version with chromaloc 2 in the code and command active.

I also removed the old ffmpeg files in the nvenc staxrip folder, since 4.03 and up use other ffmpeg dll's.

Not sure what was the problem ( probably leaving the old ffmpeg dll's among with the new? )

techok commented 6 years ago

pwacooijmans: aq strength is listed as hevc options, when you do not set is automatic?

By the way why is not possible to encode h264 in 10-bit with HDR metadata? The only way i can do this is by using the CPU H264 encoder and patch the hdr metadata with mkvtoolnix ( as youtube explains ). It would be good to also have h.264 10-bit encoding by NvEncC64.

pwacooijmans commented 6 years ago

--aq-strength [H264] set aq strength (weak 1 - 15 strong) default: 0 = auto this is version 4.06 Auto is the only default correct if you use --aq for h265 instead of --aq-temporal --aq-strength x for h264

pwacooijmans commented 6 years ago

So best NOT to add it incase of --aq for h265

pwacooijmans commented 6 years ago

Posted the entire help file here for you to read for 4.06 it's in an edited post which you won't receive by e-mail...so look at https://github.com/rigaya/NVEnc/issues/56

techok commented 6 years ago

Why you used videoformat PAL ? that PAL is not the HDR PAL as far as i know.

pwacooijmans commented 6 years ago

It's because a LOT of tv's accept EVERYTHING but undefined...for more info see this picture...I live in holland so that's why I added pal.

Op za 16 jun. 2018 om 03:53 schreef techok notifications@github.com:

Why you used videoformat PAL ? that PAL is not the HDR PAL as far as i know.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rigaya/NVEnc/issues/56#issuecomment-397778471, or mute the thread https://github.com/notifications/unsubscribe-auth/AdX5NDUCpoJr14400BB18ozx2jELOw15ks5t9GUvgaJpZM4Ujm-1 .

pwacooijmans commented 6 years ago

In short if it doesn't specifically say it ALSO support UNDEF next to PAL SECAM etc....then you can be about 99% sure it doesn't.

Op zo 17 jun. 2018 om 12:47 schreef Pieter Cooijmans <pwacooijmans@gmail.com

:

It's because a LOT of tv's accept EVERYTHING but undefined...for more info see this picture...I live in holland so that's why I added pal.

Op za 16 jun. 2018 om 03:53 schreef techok notifications@github.com:

Why you used videoformat PAL ? that PAL is not the HDR PAL as far as i know.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rigaya/NVEnc/issues/56#issuecomment-397778471, or mute the thread https://github.com/notifications/unsubscribe-auth/AdX5NDUCpoJr14400BB18ozx2jELOw15ks5t9GUvgaJpZM4Ujm-1 .

pwacooijmans commented 6 years ago

furthermore your tv may be a model that is limited to the old blu ray standard but CAN support 4K hdr in that case you better look up the maximum AVERAGE bitrate in the manual of your tv that it can support 4K hdr at....when it's 25000 for instance....which is pretty common for these types of tvs then you HAVE to use vbr instead of vbr-hq to get it working. Why you may wonder....well ONLY vbr garantees to say up to OR below the set bit rate for a 2 second gop. The difference with vbr-hq is that it does a second pass that ups the ENTIRE movie to an average you set, meaning you WILL go WAAAY over that bitrate for a lot of the time...due to for instance the ending of the movie being very low bitrate...so rest goes up a lot. Since your tv can't possibly load the entire movie into ram...that will not work for a long time to come :)

pwacooijmans commented 6 years ago

ALSO check the maximum bitrate...also VERY important since if you go over the maximum bitrate your device can handle...same issue...you get lag. Then you may wonder how to set the level to 5.1 high tier...well that's why I miss use vbv-bufsize on purpose.....it only checks for the highest bitrate....no matter if you set level or tier or not. So this is the only possible solution with nvenc of making 4k HDR for your tv. Since the bitrate is officially too low for 4k hdr, so with a little trick you can work around that :)

pwacooijmans commented 6 years ago

and make bloody sure it's not the bitrate of the ENTIRE movie, but ONLY of the video inside the file.

pwacooijmans commented 6 years ago

so I will repeat the settings for 4k HDR for old blu-ray standard that works on most tvs with that limitation: MAXIMUM TOTAL BITRATE (including: video, audio, subs, tags, overhead) 50000kilobits per second MAXIMUM BITRATE VIDEO ONLY: 40000kilobits per second MAXIMUM AVERAGE BITRATE VIDEO ONLY: 25000kilobits per second VBV-BUFSIZE: 160000 kilobits per second

pwacooijmans commented 6 years ago

MAXIMUM AVERAGE BITRATE is video only!

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.