search

hjdhjd / homebridge-unifi-protect

:video_camera: Complete HomeKit integration for all UniFi Protect device types with full support for most features including HomeKit Secure Video, and more. https://homebridge.io
Other
1.39k stars 83 forks source link

Unable to get vaapi or qsv to function on hardware that supports it #1008

Closed untergeek closed 8 months ago

untergeek commented 8 months ago

Describe The Problem: vainfo shows that vaapi should be available to ffmpeg for hardware encoding and decoding, but it isn't for some reason.

It should be noted that the kernel version mentioned below was installed in order to get the hardware encoding/decoding to work with Plex, which is also hosted on this machine.

To Reproduce: Clean install of homebridge + homebridge-unifi-protect on Ubuntu 22.04

Logs:

processor   : 11
vendor_id   : GenuineIntel
cpu family  : 6
model       : 167
model name  : 11th Gen Intel(R) Core(TM) i5-11600 @ 2.80GHz
stepping    : 1
microcode   : 0x59
cpu MHz     : 800.008
cache size  : 12288 KB
physical id : 0
siblings    : 12
core id     : 5
cpu cores   : 6
apicid      : 11
initial apicid  : 11
fpu     : yes
fpu_exception   : yes
cpuid level : 27
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap avx512ifma clflushopt intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid fsrm md_clear flush_l1d arch_capabilities
vmx flags   : vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid ple shadow_vmcs pml ept_mode_based_exec tsc_scaling
bugs        : spectre_v1 spectre_v2 spec_store_bypass swapgs mmio_stale_data retbleed eibrs_pbrsb gds
bogomips    : 5616.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:
[10/27/2023, 2:52:24 PM] [UniFi Protect] Using FFmpeg version: 6.0-homebridge-alpine-x86_64-static.
[10/27/2023, 2:52:24 PM] [UniFi Protect] Hardware-accelerated decoding and encoding using vaapi will be unavailable: unable to successfully validate capabilities.
[10/27/2023, 2:52:24 PM] [UniFi Protect] Hardware-accelerated decoding and encoding using qsv will be unavailable: unable to successfully validate capabilities.
vainfo
error: can't connect to X server!
libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.14 (libva 2.12.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 22.3.1 ()
vainfo: Supported profile and entrypoints
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileNone                   : VAEntrypointStats
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Simple            : VAEntrypointEncSlice
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointFEI
      VAProfileH264Main               : VAEntrypointEncSliceLP
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointFEI
      VAProfileH264High               : VAEntrypointEncSliceLP
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileJPEGBaseline           : VAEntrypointEncPicture
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264ConstrainedBaseline: VAEntrypointFEI
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointFEI
      VAProfileHEVCMain               : VAEntrypointEncSliceLP
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointEncSliceLP
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile1            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileVP9Profile3            : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointVLD
      VAProfileHEVCMain12             : VAEntrypointEncSlice
      VAProfileHEVCMain422_10         : VAEntrypointVLD
      VAProfileHEVCMain422_10         : VAEntrypointEncSlice
      VAProfileHEVCMain422_12         : VAEntrypointVLD
      VAProfileHEVCMain422_12         : VAEntrypointEncSlice
      VAProfileHEVCMain444            : VAEntrypointVLD
      VAProfileHEVCMain444            : VAEntrypointEncSliceLP
      VAProfileHEVCMain444_10         : VAEntrypointVLD
      VAProfileHEVCMain444_10         : VAEntrypointEncSliceLP
      VAProfileHEVCMain444_12         : VAEntrypointVLD
      VAProfileHEVCSccMain            : VAEntrypointVLD
      VAProfileHEVCSccMain            : VAEntrypointEncSliceLP
      VAProfileHEVCSccMain10          : VAEntrypointVLD
      VAProfileHEVCSccMain10          : VAEntrypointEncSliceLP
      VAProfileHEVCSccMain444         : VAEntrypointVLD
      VAProfileHEVCSccMain444         : VAEntrypointEncSliceLP
      VAProfileAV1Profile0            : VAEntrypointVLD
      VAProfileHEVCSccMain444_10      : VAEntrypointVLD
      VAProfileHEVCSccMain444_10      : VAEntrypointEncSliceLP

Homebridge Configuration:

        {
            "controllers": [
                {
                    "address": "REDACTED",
                    "defaultDoorbellMessage": "Hello There!",
                    "password": "REDACTED",
                    "username": "REDACTED"
                }
            ],
            "name": "UniFi Protect",
            "options": [
                "Enable.Device.SyncName.74ACB93BA35B",
                "Enable.Audio.Filter.Noise.E063DA00AE69",
                "Enable.Audio.Filter.Noise.FftNr.E063DA00AE69.90",
                "Enable.Audio.Filter.Noise.HighPass.E063DA00AE69.200",
                "Enable.Audio.Filter.Noise.LowPass.E063DA00AE69.1000",
                "Enable.Video.HKSV.Recording.MaxDuration.E063DA00AE69.60",
                "Enable.Video.HKSV.Recording.Switch.E063DA00AE69",
                "Enable.Video.Transcode.E063DA00AE69",
                "Enable.Video.Transcode.Hardware.E063DA00AE69",
                "Enable.SecuritySystem.Alarm.E063DA00AE69",
                "Enable.Motion.OccupancySensor.E063DA00AE69",
                "Enable.Motion.SmartDetect.E063DA00AE69",
                "Enable.Motion.Switch.E063DA00AE69",
                "Enable.Log.HKSV.E063DA00AE69",
                "Enable.Log.Motion.E063DA00AE69",
                "Enable.Doorbell.Trigger.E063DA00AE69",
                "Enable.Doorbell.PhysicalChime.E063DA00AE69",
                "Enable.Motion.SmartDetect.74ACB93BA35B",
                "Enable.Video.HKSV.Record.Only.High.74ACB93BA35B",
                "Enable.Video.Transcode.Hardware.74ACB93BA35B"
            ],
            "platform": "UniFi Protect"
        }

Screenshots: N/A

Environment:

hjdhjd commented 8 months ago

This will be a bit long, but hopefully helpful…

Thanks for the report! First, other than on macOS, hardware acceleration should be considered a best-effort/barely-supported feature. I need to get a lot more field testing in before I’m comfortable saying “everyone should use this by default”. Given the significant disparities in user hardware…I hope you can appreciate the task at hand.

Which leads to QSV. I chose to add support for QSV because it has a wide footprint across the CPU/GPU landscape and would provide the biggest “user bang” for the effort.

Which leads to FFmpeg. HBUP includes a bundled version of FFmpeg, as do most of the Homebridge solutions out there - ffmpeg-for-homebridge. That’s what provides the foundation here. We’ve been experimenting with adding QSV support there and are continuing to run into some challenges and gaps along the way. While the version of FFmpeg that comes bundled with HBUP provides some of the hooks for QSV, it doesn’t currently provide all of them.

I’m working through that.

In the meantime…if you’d like to get HBUP working with QSV hardware acceleration, you’ll need to compile/acquire your own version of FFmpeg that includes support for fdk_aac and QSV (either MFX or VPL support). Once you do, you can tell HBUP about the path to your version of FFmpeg and it will use that instead of what’s bundled (in the HBUP webUI: settings -> additional settings -> video processor).

Hope that helps…I know it’s an imperfect solution, but that’s the best I can do at the moment here.

untergeek commented 8 months ago

Thanks for the prompt response. I get how hard it is to guarantee feature continuity with a statically compiled binary.

I will see about compiling my own FFmpeg and report the results back here. Thanks for the suggestion!

untergeek commented 8 months ago

Super confused here:

Compiled ffmpeg and test shows which hardware acceleration methods are enabled:

❯ /usr/local/ffmpeg/bin/ffmpeg -hwaccels 2>/dev/null
Hardware acceleration methods:
vaapi
qsv
drm
opencl
❯ /usr/local/ffmpeg/bin/ffmpeg -encoders 2>/dev/null | grep fdk
 A....D libfdk_aac           Fraunhofer FDK AAC (codec aac)
❯ /usr/local/ffmpeg/bin/ffmpeg -encoders 2>/dev/null | grep vaapi
 V....D av1_vaapi            AV1 (VAAPI) (codec av1)
 V....D h264_vaapi           H.264/AVC (VAAPI) (codec h264)
 V....D hevc_vaapi           H.265/HEVC (VAAPI) (codec hevc)
 V....D mjpeg_vaapi          MJPEG (VAAPI) (codec mjpeg)
 V....D mpeg2_vaapi          MPEG-2 (VAAPI) (codec mpeg2video)
 V....D vp8_vaapi            VP8 (VAAPI) (codec vp8)
 V....D vp9_vaapi            VP9 (VAAPI) (codec vp9)

And sure enough, this test also shows it should work:

/usr/local/ffmpeg/bin/ffmpeg -vaapi_device /dev/dri/renderD128
ffmpeg version N-112534-ge5f774268a Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr/local/ffmpeg --extra-cflags=-I/opt/intel/mediasdk/include --extra-ldflags=-L/opt/intel/mediasdk/lib --extra-ldflags=-L/opt/intel/mediasdk/plugins --enable-libmfx --enable-vaapi --enable-opencl --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-gpl --cpu=native --enable-libfdk-aac --enable-libx264 --enable-libx265 --extra-libs=-lpthread --enable-nonfree
  libavutil      58. 28.100 / 58. 28.100
  libavcodec     60. 30.102 / 60. 30.102
  libavformat    60. 15.101 / 60. 15.101
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9. 11.100 /  9. 11.100
  libswscale      7.  4.100 /  7.  4.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

And when I link to this compiled version of ffmpeg, it actually now says that not only are vaapi and qsv not working, but drm and opencl as well (drm and opencl are not mentioned in the bundled version, so may not have been compiled in):

[10/28/2023, 10:01:23 AM] [UniFi Protect] Using FFmpeg version: N-112534-ge5f774268a.
[10/28/2023, 10:01:23 AM] [UniFi Protect] Hardware-accelerated decoding and encoding using vaapi will be unavailable: unable to successfully validate capabilities.
[10/28/2023, 10:01:23 AM] [UniFi Protect] Hardware-accelerated decoding and encoding using qsv will be unavailable: unable to successfully validate capabilities.
[10/28/2023, 10:01:23 AM] [UniFi Protect] Hardware-accelerated decoding and encoding using drm will be unavailable: unable to successfully validate capabilities.
[10/28/2023, 10:01:23 AM] [UniFi Protect] Hardware-accelerated decoding and encoding using opencl will be unavailable: unable to successfully validate capabilities.

I've tried manually specifying h264_vaapi as the codec and this is what came back in debug logging:

[10/28/2023, 10:13:37 AM] [UniFi Protect] UDM-Pro [UDM-PRO] Front Door [G4 Doorbell]: FFmpeg process ended unexpectedly with an exit code of 8.
[10/28/2023, 10:13:37 AM] [UniFi Protect] UDM-Pro [UDM-PRO] Front Door [G4 Doorbell]: FFmpeg (N-112534-ge5f774268a) command that errored out was: /usr/local/ffmpeg/bin/ffmpeg -hide_banner -nostats -fflags +discardcorrupt -probesize 16384 -max_delay 500000 -r 30 -rtsp_transport tcp -i rtsps://REDACTED:7441/SP7i0WBMe2Px1IDk?enableSrtp -map 0:v:0 -c:v h264_vaapi -preset veryfast -profile:v main -level:v 4.0 -noautoscale -bf 0 -filter:v format=yuvj420p, scale=-2:min(ih\,480) -g:v 60 -bufsize 328k -maxrate 228k -crf 20 -payload_type 99 -ssrc 13805113 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params Rcg40hIRzsWOsYZsi/nVPM0AP/GG0amyM1EuTlcw srtp://REDACTED:52842?rtcpport=52842&pkt_size=564 -map 0:a:0? -acodec libfdk_aac -afterburner 1 -eld_sbr 1 -eld_v2 1 -profile:a 38 -flags +global_header -f null -ar 24k -b:a 24k -bufsize 48k -ac 1 -af afftdn=nt=w:om=o:tn=1:tr=1:nr=90, highpass=f=200, lowpass=f=1000 -payload_type 110 -ssrc 6357058 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params ELhI8UI1LFIVB5GRZpeCKes2cHr8Ef9k8vkEfGr/ srtp://REDACTED:58719?rtcpport=58719&pkt_size=188 -loglevel level+verbose

I don't need hw acceleration to work, so I will happily go back to using the bundled ffmpeg and let it work. I figured this was worth reporting here though for future reference.

untergeek commented 8 months ago

I tried using the h264_qsv codec at the command-line:

❯ /usr/local/ffmpeg/bin/ffmpeg -vaapi_device /dev/dri/renderD128 -i /plex/tmp/SOURCE.mp4  -vcodec h264_qsv buh.mp4
ffmpeg version N-112534-ge5f774268a Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr/local/ffmpeg --extra-cflags=-I/opt/intel/mediasdk/include --extra-ldflags=-L/opt/intel/mediasdk/lib --extra-ldflags=-L/opt/intel/mediasdk/plugins --enable-libmfx --enable-vaapi --enable-opencl --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-gpl --cpu=native --enable-libfdk-aac --enable-libx264 --enable-libx265 --extra-libs=-lpthread --enable-nonfree
  libavutil      58. 28.100 / 58. 28.100
  libavcodec     60. 30.102 / 60. 30.102
  libavformat    60. 15.101 / 60. 15.101
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9. 11.100 /  9. 11.100
  libswscale      7.  4.100 /  7.  4.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/plex/tmp/SOURCE.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
    creation_time   : 2023-02-20T05:12:11.000000Z
  Duration: 01:36:58.98, start: 0.000000, bitrate: 5745 kb/s
  Stream #0:0[0x1](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 126 kb/s (default)
    Metadata:
      creation_time   : 2023-02-20T05:12:13.000000Z
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Metadata:
      creation_time   : 2023-02-20T05:14:55.000000Z
      vendor_id       : [0][0][0][0]
    Side data:
      audio service type: main
  Stream #0:2[0x3](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1912x1076 [SAR 1:1 DAR 478:269], 5231 kb/s, 23.98 fps, 23.98 tbr, 24k tbn (default)
    Metadata:
      creation_time   : 2023-02-20T05:16:01.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : H.264
  Stream #0:3[0x4](eng): Subtitle: eia_608 (c608 / 0x38303663), 0 kb/s
    Metadata:
      creation_time   : 2023-02-20T05:17:21.000000Z
File 'buh.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:2 -> #0:0 (h264 (native) -> h264 (h264_qsv))
  Stream #0:0 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
Output #0, mp4, to 'buh.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
    encoder         : Lavf60.15.101
  Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), nv12(tv, bt709, progressive), 1912x1076 [SAR 1:1 DAR 478:269], q=2-31, 1000 kb/s, 23.98 fps, 24k tbn (default)
    Metadata:
      creation_time   : 2023-02-20T05:16:01.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.30.102 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: N/A
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2023-02-20T05:12:13.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.30.102 aac
[out#0/mp4 @ 0x5592b37b22c0] video:38843kB audio:5251kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.489895%
frame= 8051 fps=247 q=35.0 Lsize=   44310kB time=00:05:37.23 bitrate=1076.4kbits/s dup=1 drop=0 speed=10.3x
[aac @ 0x5592b36201c0] Qavg: 2202.490
Exiting normally, received signal 2.
❯ file buh.mp4
buh.mp4: ISO Media, MP4 Base Media v1 [ISO 14496-12:2003]
untergeek commented 8 months ago

And for giggles, I tried the same with the bundled ffmpeg. It errored out:

[h264_qsv @ 0x7f0a0870d400] Error creating a MFX session: -9.
[vost#0:0/h264_qsv @ 0x7f0a08816040] Error initializing output stream: Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
[aac @ 0x7f0a07b8b040] Qavg: 65536.000
[aac @ 0x7f0a07b8b040] 2 frames left in the queue on closing
Conversion failed!
untergeek commented 8 months ago

So, I'm back to using bundled. Hope these reports are helpful. If you want me to try settings or edits to files with what seems like a QSV functional version of ffmpeg, I'm game. Just let me know.

hjdhjd commented 8 months ago

Super helpful. Can you try the same test command line without specifying the -vaapi_device parameter using your own FFmpeg (/usr/local/bin in your case)?

untergeek commented 8 months ago

Yeah. I should have removed that. It doesn't change the output, though:

/usr/local/ffmpeg/bin/ffmpeg -i /plex/tmp/SOURCE.mp4  -vcodec h264_qsv buh.mp4
ffmpeg version N-112534-ge5f774268a Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr/local/ffmpeg --extra-cflags=-I/opt/intel/mediasdk/include --extra-ldflags=-L/opt/intel/mediasdk/lib --extra-ldflags=-L/opt/intel/mediasdk/plugins --enable-libmfx --enable-vaapi --enable-opencl --disable-debug --enable-libvorbis --enable-libvpx --enable-libdrm --enable-gpl --cpu=native --enable-libfdk-aac --enable-libx264 --enable-libx265 --extra-libs=-lpthread --enable-nonfree
  libavutil      58. 28.100 / 58. 28.100
  libavcodec     60. 30.102 / 60. 30.102
  libavformat    60. 15.101 / 60. 15.101
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9. 11.100 /  9. 11.100
  libswscale      7.  4.100 /  7.  4.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/plex/tmp/SOURCE.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
    creation_time   : 2023-02-20T05:12:11.000000Z
  Duration: 01:36:58.98, start: 0.000000, bitrate: 5745 kb/s
  Stream #0:0[0x1](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 126 kb/s (default)
    Metadata:
      creation_time   : 2023-02-20T05:12:13.000000Z
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Metadata:
      creation_time   : 2023-02-20T05:14:55.000000Z
      vendor_id       : [0][0][0][0]
    Side data:
      audio service type: main
  Stream #0:2[0x3](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1912x1076 [SAR 1:1 DAR 478:269], 5231 kb/s, 23.98 fps, 23.98 tbr, 24k tbn (default)
    Metadata:
      creation_time   : 2023-02-20T05:16:01.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : H.264
  Stream #0:3[0x4](eng): Subtitle: eia_608 (c608 / 0x38303663), 0 kb/s
    Metadata:
      creation_time   : 2023-02-20T05:17:21.000000Z
Stream mapping:
  Stream #0:2 -> #0:0 (h264 (native) -> h264 (h264_qsv))
  Stream #0:0 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
Output #0, mp4, to 'buh.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
    encoder         : Lavf60.15.101
  Stream #0:0(eng): Video: h264 (avc1 / 0x31637661), nv12(tv, bt709, progressive), 1912x1076 [SAR 1:1 DAR 478:269], q=2-31, 1000 kb/s, 23.98 fps, 24k tbn (default)
    Metadata:
      creation_time   : 2023-02-20T05:16:01.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.30.102 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: N/A
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2023-02-20T05:12:13.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.30.102 aac
frame= 3464 fps=256 q=45.0 size=   18688kB time=00:02:26.15 bitrate=1047.5kbits/s dup=1 drop=0 speed=10.8x

(I copy/pasted before waiting for it to finish)

untergeek commented 8 months ago

I'm running some experiments. The QSV is definitely working.

Preliminary results of libx264 have the same source file averaging 60fps The same source file using h264_qsv is hovering between 245fps and 249fps

I will share the comparative results after the fact.

hjdhjd commented 8 months ago

Glad to hear it…if you don’t mind, can you ping me on the Homebridge Discord? I would love to collaborate with you on this and try to bring this to the masses.

untergeek commented 8 months ago

Sure thing. I will hop over when I can. I'm about to head out.

So, here's the performance with the libx264:

Average fps: 56

frame=139517 fps= 56 q=-1.0 Lsize= 3938290kB time=01:36:58.94 bitrate=5544.4kbits/s dup=1 drop=0 speed=2.32x
[libx264 @ 0x55643ffc5c00] frame I:1546  Avg QP:20.49  size: 98797
[libx264 @ 0x55643ffc5c00] frame P:42247 Avg QP:23.32  size: 43923
[libx264 @ 0x55643ffc5c00] frame B:95724 Avg QP:25.16  size: 20124
[libx264 @ 0x55643ffc5c00] consecutive B-frames:  4.5% 11.4%  1.9% 82.1%
[libx264 @ 0x55643ffc5c00] mb I  I16..4:  5.2% 85.0%  9.7%
[libx264 @ 0x55643ffc5c00] mb P  I16..4:  1.1% 13.2%  0.7%  P16..4: 54.3% 13.9%  5.5%  0.0%  0.0%    skip:11.3%
[libx264 @ 0x55643ffc5c00] mb B  I16..4:  0.2%  2.0%  0.1%  B16..8: 48.5%  5.5%  0.7%  direct: 5.2%  skip:37.9%  L0:49.3% L1:47.3% BI: 3.4%
[libx264 @ 0x55643ffc5c00] 8x8 transform intra:87.7% inter:86.5%
[libx264 @ 0x55643ffc5c00] coded y,uvDC,uvAC intra: 74.3% 58.9% 8.6% inter: 27.3% 16.1% 0.1%
[libx264 @ 0x55643ffc5c00] i16 v,h,dc,p: 22% 21%  6% 51%
[libx264 @ 0x55643ffc5c00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 17% 20%  6%  8%  8%  8%  8%  8%
[libx264 @ 0x55643ffc5c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 20% 12%  6%  9%  8%  8%  6%  5%
[libx264 @ 0x55643ffc5c00] i8c dc,h,v,p: 57% 20% 18%  5%
[libx264 @ 0x55643ffc5c00] Weighted P-Frames: Y:3.3% UV:0.8%
[libx264 @ 0x55643ffc5c00] ref P L0: 54.2% 13.4% 22.6%  9.6%  0.3%
[libx264 @ 0x55643ffc5c00] ref B L0: 81.5% 13.5%  5.0%
[libx264 @ 0x55643ffc5c00] ref B L1: 95.7%  4.3%
[libx264 @ 0x55643ffc5c00] kb/s:5409.41
[aac @ 0x55643fe88080] Qavg: 787.807

Will start up the h264_qsv on my way out.

untergeek commented 8 months ago

And the h264_qsv performance:

Average fps: 249

frame=139517 fps=249 q=26.0 Lsize=  773324kB time=01:36:58.94 bitrate=1088.7kbits/s dup=1 drop=0 speed=10.4x
[aac @ 0x560bbc889f40] Qavg: 787.807

I had the presence of mind to run this with time: 

2077.50s user 73.58s system 383% cpu 9:20.48 total

So much faster. Will join Discord soon.

hjdhjd commented 8 months ago

I’m not worried in the least about performance of hardware acceleration. That I already know will be fine through my own testing…I’m more keen to ensure it gets properly working across a broad set of user environments. Hence wanting to connect. The stats are nice…but candidly…already knew all that. 😄

Now let’s talk about getting it working.

untergeek commented 8 months ago

Okay. I'm on Discord now as Untergeek

github-actions[bot] commented 8 months ago

This issue is locked to prevent necroposting on closed issues. Please create a new issue for related support requests, bug reports, or feature suggestions.