search

JuliaIO / VideoIO.jl

Reading and writing of video files in Julia via ffmpeg
https://juliaio.github.io/VideoIO.jl/stable
Other
126 stars 53 forks source link

Add optional nvidia hardware-accelerated h264_nvenc encoding #251

Closed IanButterworth closed 4 years ago

IanButterworth commented 4 years ago

Hoping to add hardware accelerated encoding, as per https://trac.ffmpeg.org/wiki/HWAccelIntro

Dependent on an updated FFMPEG_jll via https://github.com/JuliaPackaging/Yggdrasil/pull/1384 and https://github.com/JuliaIO/FFMPEG.jl/pull/32

codecov[bot] commented 4 years ago

Codecov Report

Merging #251 into master will not change coverage. The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #251   +/-   ##
=======================================
  Coverage   77.09%   77.09%           
=======================================
  Files          14       14           
  Lines         598      598           
=======================================
  Hits          461      461           
  Misses        137      137
Impacted Files Coverage Δ
src/encoding.jl 90.97% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update ce0ed3c...74f065e. Read the comment docs.

IanButterworth commented 4 years ago

WIP:

using FFMPEG
#FFMPEG.exe(`-h encoder=libx264`)
FFMPEG.exe(`-h encoder=h264_nvenc`)
ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8.1.0 (GCC)
  configuration: --enable-cross-compile --cross-prefix=/opt/x86_64-linux-gnu/bin/x86_64-linux-gnu- --arch=x86_64 --target-os=linux --cc=cc --cxx=c++ --dep-cc=cc --ar=ar --nm=nm --objcc=objc --sysinclude=/workspace/destdir/include --pkg-config=/usr/bin/pkg-config --pkg-config-flags=--static --prefix=/workspace/destdir --sysroot=/opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root --extra-libs=-lpthread --enable-gpl --enable-version3 --enable-nonfree --disable-static --enable-shared --enable-pic --disable-debug --disable-doc --enable-avresample --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libx264 --enable-libx265 --enable-libvpx --enable-encoders --enable-decoders --enable-muxers --enable-demuxers --enable-parsers --enable-openssl --disable-schannel --extra-cflags=-I/workspace/destdir/include --extra-ldflags=-L/workspace/destdir/lib --enable-nvenc --enable-cuda-llvm
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Encoder h264_nvenc [NVIDIA NVENC H.264 encoder]:
    General capabilities: delay hardware 
    Threading capabilities: none
    Supported hardware devices: cuda cuda 
    Supported pixel formats: yuv420p nv12 p010le yuv444p p016le yuv444p16le bgr0 rgb0 cuda
h264_nvenc AVOptions:
  -preset            <int>        E..V...... Set the encoding preset (from 0 to 11) (default medium)
     default         0            E..V...... 
     slow            1            E..V...... hq 2 passes
     medium          2            E..V...... hq 1 pass
     fast            3            E..V...... hp 1 pass
     hp              4            E..V...... 
     hq              5            E..V...... 
     bd              6            E..V...... 
     ll              7            E..V...... low latency
     llhq            8            E..V...... low latency hq
     llhp            9            E..V...... low latency hp
     lossless        10           E..V...... 
     losslesshp      11           E..V...... 
  -profile           <int>        E..V...... Set the encoding profile (from 0 to 3) (default main)
     baseline        0            E..V...... 
     main            1            E..V...... 
     high            2            E..V...... 
     high444p        3            E..V...... 
  -level             <int>        E..V...... Set the encoding level restriction (from 0 to 51) (default auto)
     auto            0            E..V...... 
     1               10           E..V...... 
     1.0             10           E..V...... 
     1b              9            E..V...... 
     1.0b            9            E..V...... 
     1.1             11           E..V...... 
     1.2             12           E..V...... 
     1.3             13           E..V...... 
     2               20           E..V...... 
     2.0             20           E..V...... 
     2.1             21           E..V...... 
     2.2             22           E..V...... 
     3               30           E..V...... 
     3.0             30           E..V...... 
     3.1             31           E..V...... 
     3.2             32           E..V...... 
     4               40           E..V...... 
     4.0             40           E..V...... 
     4.1             41           E..V...... 
     4.2             42           E..V...... 
     5               50           E..V...... 
     5.0             50           E..V...... 
     5.1             51           E..V...... 
  -rc                <int>        E..V...... Override the preset rate-control (from -1 to INT_MAX) (default -1)
     constqp         0            E..V...... Constant QP mode
     vbr             1            E..V...... Variable bitrate mode
     cbr             2            E..V...... Constant bitrate mode
     vbr_minqp       8388612      E..V...... Variable bitrate mode with MinQP (deprecated)
     ll_2pass_quality 8388616      E..V...... Multi-pass optimized for image quality (deprecated)
     ll_2pass_size   8388624      E..V...... Multi-pass optimized for constant frame size (deprecated)
     vbr_2pass       8388640      E..V...... Multi-pass variable bitrate mode (deprecated)
     cbr_ld_hq       8            E..V...... Constant bitrate low delay high quality mode
     cbr_hq          16           E..V...... Constant bitrate high quality mode
     vbr_hq          32           E..V...... Variable bitrate high quality mode
  -rc-lookahead      <int>        E..V...... Number of frames to look ahead for rate-control (from 0 to INT_MAX) (default 0)
  -surfaces          <int>        E..V...... Number of concurrent surfaces (from 0 to 64) (default 0)
  -cbr               <boolean>    E..V...... Use cbr encoding mode (default false)
  -2pass             <boolean>    E..V...... Use 2pass encoding mode (default auto)
  -gpu               <int>        E..V...... Selects which NVENC capable GPU to use. First GPU is 0, second is 1, and so on. (from -2 to INT_MAX) (default any)
     any             -1           E..V...... Pick the first device available
     list            -2           E..V...... List the available devices
  -delay             <int>        E..V...... Delay frame output by the given amount of frames (from 0 to INT_MAX) (default INT_MAX)
  -no-scenecut       <boolean>    E..V...... When lookahead is enabled, set this to 1 to disable adaptive I-frame insertion at scene cuts (default false)
  -forced-idr        <boolean>    E..V...... If forcing keyframes, force them as IDR frames. (default false)
  -b_adapt           <boolean>    E..V...... When lookahead is enabled, set this to 0 to disable adaptive B-frame decision (default true)
  -spatial-aq        <boolean>    E..V...... set to 1 to enable Spatial AQ (default false)
  -spatial_aq        <boolean>    E..V...... set to 1 to enable Spatial AQ (default false)
  -temporal-aq       <boolean>    E..V...... set to 1 to enable Temporal AQ (default false)
  -temporal_aq       <boolean>    E..V...... set to 1 to enable Temporal AQ (default false)
  -zerolatency       <boolean>    E..V...... Set 1 to indicate zero latency operation (no reordering delay) (default false)
  -nonref_p          <boolean>    E..V...... Set this to 1 to enable automatic insertion of non-reference P-frames (default false)
  -strict_gop        <boolean>    E..V...... Set 1 to minimize GOP-to-GOP rate fluctuations (default false)
  -aq-strength       <int>        E..V...... When Spatial AQ is enabled, this field is used to specify AQ strength. AQ strength scale is from 1 (low) - 15 (aggressive) (from 1 to 15) (default 8)
  -cq                <float>      E..V...... Set target quality level (0 to 51, 0 means automatic) for constant quality mode in VBR rate control (from 0 to 51) (default 0)
  -aud               <boolean>    E..V...... Use access unit delimiters (default false)
  -bluray-compat     <boolean>    E..V...... Bluray compatibility workarounds (default false)
  -init_qpP          <int>        E..V...... Initial QP value for P frame (from -1 to 51) (default -1)
  -init_qpB          <int>        E..V...... Initial QP value for B frame (from -1 to 51) (default -1)
  -init_qpI          <int>        E..V...... Initial QP value for I frame (from -1 to 51) (default -1)
  -qp                <int>        E..V...... Constant quantization parameter rate control method (from -1 to 51) (default -1)
  -weighted_pred     <int>        E..V...... Set 1 to enable weighted prediction (from 0 to 1) (default 0)
  -coder             <int>        E..V...... Coder type (from -1 to 2) (default default)
     default         -1           E..V...... 
     auto            0            E..V...... 
     cabac           1            E..V...... 
     cavlc           2            E..V...... 
     ac              1            E..V...... 
     vlc             2            E..V...... 
  -b_ref_mode        <int>        E..V...... Use B frames as references (from 0 to 2) (default disabled)
     disabled        0            E..V...... B frames will not be used for reference
     each            1            E..V...... Each B frame will be used for reference
     middle          2            E..V...... Only (number of B frames)/2 will be used for reference
  -a53cc             <boolean>    E..V...... Use A53 Closed Captions (if available) (default true)
  -dpb_size          <int>        E..V...... Specifies the DPB size used for encoding (0 means automatic) (from 0 to INT_MAX) (default 0)

using VideoIO, ImageCore
n = 300
imgstack = map(x->rand(RGB{N0f8}, 2048, 1536), 1:n)
props = [:priv_data => ("crf"=>"22","preset"=>"medium")]
for codec in ["libx264", "h264_nvenc"]
    encoder = prepareencoder(imgstack[1], framerate=60, AVCodecContextProperties=props, codec_name=codec)
    io = Base.open("temp.stream","w")
    t = @elapsed for i in 1:n
        appendencode!(encoder, io, imgstack[i], i)
    end
    @info "$codec fps: $(n/t)"
    finishencode!(encoder, io)
    close(io)
end

Currently errors on the nvenc codec with Could not open codec: Return code -38 even though this process is using yuv420p and medium which are both supported

IanButterworth commented 4 years ago

Merging to allow the new FFMPEG_jll version (which loads slightly faster). The hardware acceleration codec integration needs more figuring out. It seems CUDA needs to be linked at runtime somehow, but that's experimental for now