Intel Quick Sync Support

[mitchross]

Hello - I would like to know the commands to use Intel Quick Sync with this project

https://jellyfin.org/docs/general/administration/hardware-acceleration/#intel-quicksync https://deploy-preview-4055--frigate-docs.netlify.app/configuration/hardware_acceleration

I previously opened up... https://github.com/mrlt8/docker-wyze-bridge/issues/548

[mitchross]

@mrlt8 I rewrote my docker - intel - qsv container. I know FFMpeg with qsv works, because this command works ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf hwdownload,format=nv12 -pix_fmt yuv420p output.yuv

When I run this ffmpeg command

- name: FFMPEG_CMD
  value: 'ffmpeg -loglevel verbose -f h264 -hwaccel qsv -hwaccel_output_format qsv -i - -c:v h264_qsv -rtsp_transport tcp -f rtsp rtsp://0.0.0.0:8554/{cam_name}'

  I get this error in the logs..... 

Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv)) [h264 @ 0x55c29e745100] Reinit context to 1920x1088, pix_fmt: yuv420p [graph 0 input from stream 0:0 @ 0x55c29ef16b40] w:1920 h:1080 pixfmt:yuv420p tb:1/1200000 fr:20/1 sar:0/1 [auto_scale_0 @ 0x55c29ef18d80] w:iw h:ih flags:'' interl:0 [format @ 0x55c29ef174c0] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format' [auto_scale_0 @ 0x55c29ef18d80] w:1920 h:1080 fmt:yuv420p sar:0/1 -> w:1920 h:1080 fmt:nv12 sar:0/1 flags:0x0 Last message repeated 3 times [h264_qsv @ 0x55c29e547bc0] Using device qsv0 (type qsv) with h264_qsv encoder. [h264_qsv @ 0x55c29e547bc0] Encoder: input is system memory surface [h264_qsv @ 0x55c29e547bc0] Using the constant bitrate (CBR) ratecontrol method [h264_qsv @ 0x55c29e547bc0] MFMode:2 [h264_qsv @ 0x55c29e547bc0] Selected ratecontrol mode is unsupported [h264_qsv @ 0x55c29e547bc0] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters. Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height [AVIOContext @ 0x55c29e5329c0] Statistics: 1049163 bytes read, 0 seeks Conversion failed! [dog-cam-2] FFMPEG stopped [kitchen] FFMPEG stopped [h264 @ 0x56286f856c80] max_analyze_duration 5000000 reached at 5000000 microseconds st:0 Input #0, h264, from 'pipe:': Duration: N/A, bitrate: N/A Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(tv, unknown/bt709/unknown, progressive, left), 1728x1296, 20 fps, 20 tbr, 1200k tbn

Any thoughts?

[mrlt8]

[h264_qsv @ 0x55c29e547bc0] Selected ratecontrol mode is unsupported

Does this help -b:v 3000k? That's what we use with h264_v4l2m2m and h264_nvenc. Each encoder seems to require a little tweaking.

[mitchross]

Same error with ->   value: 'ffmpeg -loglevel verbose -f h264 -hwaccel qsv -hwaccel_output_format qsv -i - -c:v h264_qsv -b:v 3000k -rtsp_transport tcp -f rtsp rtsp://0.0.0.0:8554/{cam_name}'

[front-door] FFMPEG stopped [h264 @ 0x557988e0eac0] max_analyze_duration 5000000 reached at 5000000 microseconds st:0 Input #0, h264, from 'pipe:': Duration: N/A, bitrate: N/A Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(tv, bt709, progressive, left), 1920x1080 (1920x1088), 20 fps, 20 tbr, 1200k tbn [AVHWDeviceContext @ 0x557989080d40] Trying to use DRM render node for device 0, with matching kernel driver (i915). [AVHWDeviceContext @ 0x557989080d40] libva: VA-API version 1.18.0 [AVHWDeviceContext @ 0x557989080d40] libva: User requested driver 'iHD' [AVHWDeviceContext @ 0x557989080d40] libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so [AVHWDeviceContext @ 0x557989080d40] libva: Found init function __vaDriverInit_1_17 [AVHWDeviceContext @ 0x557989080d40] libva: va_openDriver() returns 0 [AVHWDeviceContext @ 0x557989080d40] Initialised VAAPI connection: version 1.18 [AVHWDeviceContext @ 0x557989080d40] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.1.2 (). [AVHWDeviceContext @ 0x557989080d40] Driver not found in known nonstandard list, using standard behaviour. [AVHWDeviceContext @ 0x557988e24940] Initialize MFX session: API version is 1.35, implementation version is 1.35 Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv)) [h264 @ 0x557988eaa2c0] Reinit context to 1920x1088, pix_fmt: yuv420p [graph 0 input from stream 0:0 @ 0x5579897b6b40] w:1920 h:1080 pixfmt:yuv420p tb:1/1200000 fr:20/1 sar:0/1 [auto_scale_0 @ 0x5579897b8dc0] w:iw h:ih flags:'' interl:0 [format @ 0x5579897b7500] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format' [auto_scale_0 @ 0x5579897b8dc0] w:1920 h:1080 fmt:yuv420p sar:0/1 -> w:1920 h:1080 fmt:nv12 sar:0/1 flags:0x0 Last message repeated 3 times [h264_qsv @ 0x557988e373c0] Using device qsv0 (type qsv) with h264_qsv encoder. [h264_qsv @ 0x557988e373c0] Encoder: input is system memory surface [h264_qsv @ 0x557988e373c0] Using the variable bitrate (VBR) ratecontrol method [h264_qsv @ 0x557988e373c0] MFMode:2 [h264_qsv @ 0x557988e373c0] Selected ratecontrol mode is unsupported [h264_qsv @ 0x557988e373c0] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters. Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height [AVIOContext @ 0x557988e1f800] Statistics: 716685 bytes read, 0 seeks Conversion failed! [h264 @ 0x5633bcbf91c0] Reinit context to 1920x1088, pix_fmt: yuv420p [h264 @ 0x559f50d96ac0] max_analyze_duration 5000000 reached at 5000000 microseconds st:0 Input #0, h264, from 'pipe:': Duration: N/A, bitrate: N/A Stream #0:0: Video: h264 (High), 1 reference frame, yuv420p(tv, bt709, progressive, left), 1920x1080 (1920x1088), 20 fps, 20 tbr, 1200k tbn [AVHWDeviceContext @ 0x559f50db8b80] Trying to use DRM render node for device 0, with matching kernel driver (i915). [AVHWDeviceContext @ 0x559f50db8b80] libva: VA-API version 1.18.0 [AVHWDeviceContext @ 0x559f50db8b80] libva: User requested driver 'iHD' [AVHWDeviceContext @ 0x559f50db8b80] libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so [AVHWDeviceContext @ 0x559f50db8b80] libva: Found init function __vaDriverInit_1_17 [AVHWDeviceContext @ 0x559f50db8b80] libva: va_openDriver() returns 0 [AVHWDeviceContext @ 0x559f50db8b80] Initialised VAAPI connection: version 1.18 [AVHWDeviceContext @ 0x559f50db8b80] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 23.1.2 (). [AVHWDeviceContext @ 0x559f50db8b80] Driver not found in known nonstandard list, using standard behaviour. [AVHWDeviceContext @ 0x559f50ed3ac0] Initialize MFX session: API version is 1.35, implementation version is 1.35 Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv)) [h264 @ 0x559f50e35e80] Reinit context to 1920x1088, pix_fmt: yuv420p [graph 0 input from stream 0:0 @ 0x559f517695c0] w:1920 h:1080 pixfmt:yuv420p tb:1/1200000 fr:20/1 sar:0/1 [auto_scale_0 @ 0x559f5176b840] w:iw h:ih flags:'' interl:0 [format @ 0x559f51769f80] auto-inserting filter 'auto_scale_0' between the filter 'Parsed_null_0' and the filter 'format' [auto_scale_0 @ 0x559f5176b840] w:1920 h:1080 fmt:yuv420p sar:0/1 -> w:1920 h:1080 fmt:nv12 sar:0/1 flags:0x0 Last message repeated 3 times [h264_qsv @ 0x559f50db7200] Using device qsv0 (type qsv) with h264_qsv encoder. [h264_qsv @ 0x559f50db7200] Encoder: input is system memory surface [h264_qsv @ 0x559f50db7200] Using the variable bitrate (VBR) ratecontrol method [h264_qsv @ 0x559f50db7200] MFMode:2 [h264_qsv @ 0x559f50db7200] Selected ratecontrol mode is unsupported [h264_qsv @ 0x559f50db7200] some encoding parameters are not supported by the QSV runtime. Please double check the input parameters. Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height [AVIOContext @ 0x559f50da7800] Statistics: 894074 bytes read, 0 seeks Conversion failed! [garage] FFMPEG stopped [WyzeBridge] 🎉 Connecting to WyzeCam V3 - Dog Cam 2 on 192.168.1.184 [living-room] FFMPEG stopped [WyzeBridge] 🎉 Connecting to WyzeCam Doorbell - Front door on 192.168.1.105 [WyzeBridge] 🎉 Connecting to WyzeCam V3 - Garage on 192.168.1.95 [WyzeBridge] 🎉 Connecting to WyzeCam V3 - Deck on 192.168.1.196 [shed] ☁️ Camera is connected via RELAY mode!! [shed] Stream may consume additional bandwidth! [shed] 📡 Getting 180kb/s HD stream (H264/20fps) via RELAY mode (WiFi: 71%) FW: 4.36.11.4679 🔒 (DTLS) (2/3) ffmpeg version n5.1-2-g915ef932a3-20220731 Copyright (c) 2000-2022 the FFmpeg developers built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b) configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20220731 libavutil 57. 28.100 / 57. 28.100 libavcodec 59. 37.100 / 59. 37.100 libavformat 59. 27.100 / 59. 27.100 libavdevice 59. 7.100 / 59. 7.100 libavfilter 8. 44.100 / 8. 44.100 libswscale 6. 7.100 / 6. 7.100 libswresample 4. 7.100 / 4. 7.100 libpostproc 56. 6.100 / 56. 6.100 [shed] WARNING: Skipping smaller frame at start of stream (frame_size=1) [h264 @ 0x5557950841c0] Reinit context to 1920x1088, pix_fmt: yuv420p

[mitchross]

fwiw @mrlt8 frigate's ffmpeg and docker lets me use intel qsv with no issue. My Docker image uses same ffmpeg

If it helped, frigate seems to use these params

https://github.com/blakeblackshear/frigate/blob/9e531b0b5bf34634dc6c44e7eca7e5eaf5a52a25/frigate/ffmpeg_presets.py#L67

[mrlt8]

do you need to add -vaapi_device /dev/dri/renderD128?

[mitchross]

do you need to add -vaapi_device /dev/dri/renderD128?

No dont think so, my test commands from the FFMPEG website didn't require it... Im thinking its something with this line https://github.com/blakeblackshear/frigate/blob/9e531b0b5bf34634dc6c44e7eca7e5eaf5a52a25/frigate/ffmpeg_presets.py#L113C6-L113C6

[mrlt8]

that's mostly a video filter..?

Your previous reference shows frigate using -qsv_device and then selecting the appropriate device

[mitchross]

that's mostly a video filter..?

Your previous reference shows frigate using -qsv_device and then selecting the appropriate device

Here is the command that works natively in my container...

https://trac.ffmpeg.org/wiki/Hardware/QuickSync

Theres no need to pass a device... When I run intel_gpu_top the GPU gets activated

[mrlt8]

The mrlt8/wyze-bridge:latest-hw image seems to support qsv:

 V....D libx264              libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
 V....D libx264rgb           libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
 V....D libopenh264          OpenH264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
 V....D h264_amf             AMD AMF H.264 Encoder (codec h264)
 V....D h264_nvenc           NVIDIA NVENC H.264 encoder (codec h264)
 V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
 V..... h264_v4l2m2m         V4L2 mem2mem H.264 encoder wrapper (codec h264)
 V....D h264_vaapi           H.264/AVC (VAAPI) (codec h264)

would something like this work?

- FFMPEG_CMD=ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -i - -c:v h264_qsv -b:v 4000k -vf 'format=nv12|vaapi,hwupload' -rtsp_transport tcp -f rtsp rtsp://0.0.0.0:8554/{cam_name}

or

- FFMPEG_CMD=ffmpeg -hwaccel qsv -c:v h264_qsv -i - -c:v h264_qsv -b:v 4000k -rtsp_transport tcp -f rtsp rtsp://0.0.0.0:8554/{cam_name}

[mitchross]

@mrlt8 So FFMPEG might support qsv ( intel gen 11+ ) or vaapi (sub intel 11).. but you still need the intel drivers in the docker image . Here is what my docker file looks like https://gist.github.com/mitchross/5dc213dc6d4165704f379be414044235 ... Its based on Frigate's docker file.

So the errors I reported above are based on my docker image... You can build it yourself or I can give a docker hub image...

If you take the docker file, you can shell into it.

run 'intel_gpu_top' to get GPU activity. If you download a sample mp4, then run 'ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf hwdownload,format=nv12 -pix_fmt yuv420p output.yuv' you will see GPU monitor tool spike. This 'intel_gpu_top' in the container or on the host itself.

https://trac.ffmpeg.org/wiki/Hardware/QuickSync

[mrlt8]

Some of the ENVs were moved to an external file in v2.3.5. Would something like this work? https://raw.githubusercontent.com/mrlt8/docker-wyze-bridge/dockerfiles/app/Dockerfile.qsv

[mitchross]

Some of the ENVs were moved to an external file in v2.3.5. Would something like this work? https://raw.githubusercontent.com/mrlt8/docker-wyze-bridge/dockerfiles/app/Dockerfile.qsv

Ill give this a try tonight... do you have a sample docker compose for me to try also? Just want to ensure our testing is 1:1

[mitchross]

@mrlt8 Results

Latest-hw Image ( 2.3.5 ) + "ffmpeg -hwaccel qsv -c:v h264_qsv -i - -c:v h264_qsv -b:v 4000k -rtsp_transport tcp -f rtsp rtsp://0.0.0.0:8554/{cam_name}"

Results in the error below ( which I expect as the intel drivers are not present

[WyzeBridge] 🎉 Connecting to WyzeCam V3 - Garage on 192.168.1.95
[vist#0:0/h264 @ 0x55da611288c0] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
Input #0, h264, from 'fd:':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(tv, unknown/bt709/unknown, progressive), 1728x1296, 25 fps, 40 tbr, 1200k tbn
implib-gen: libdrm.so.2: failed to load library 'libdrm.so.2' via dlopen: libdrm.so.2: cannot open shared object file: No such file or directory
ffmpeg: libdrm.so.2.init.c:61: load_library: Assertion `0 && "Assertion in generated code"' failed.
[vist#0:0/h264 @ 0x559650f28b00] WARNING: defaulting hwaccel_output_format to qsv for compatibility with old commandlines. This behaviour is DEPRECATED and will be removed in the future. Please explicitly set "-hwaccel_output_format qsv".
Input #0, h264, from 'fd:':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 1920x1080, 25 fps, 40 tbr, 1200k tbn
implib-gen: libdrm.so.2: failed to load library 'libdrm.so.2' via dlopen: libdrm.so.2: cannot open shared object file: No such file or directory
ffmpeg: libdrm.so.2.init.c:61: load_library: Assertion `0 && "Assertion in generated code"' failed.
[wyze-doorbell] FFMPEG stopped

Now If I use the command "ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -i - -c:v h264_qsv -b:v 4000k -vf 'format=nv12|vaapi,hwupload' -rtsp_transport tcp -f rtsp rtsp://0.0.0.0:8554/{cam_name}"

I get

  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20230625
  libavutil      58. 13.101 / 58. 13.101
  libavcodec     60. 21.100 / 60. 21.100
  libavformat    60.  9.100 / 60.  9.100
  libavdevice    60.  2.100 / 60.  2.100
  libavfilter     9.  8.102 /  9.  8.102
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
implib-gen: libX11.so.6: failed to load library 'libX11.so.6' via dlopen: libX11.so.6: cannot open shared object file: No such file or directory
ffmpeg: libX11.so.6.init.c:61: load_library: Assertion `0 && "Assertion in generated code"' failed.
[living-room] WARNING: Skipping smaller frame at start of stream (frame_size=1)
[living-room] FFMPEG stopped

Next I tried your new docker image, I had to hard code the URL for MTX, I think something is busted with the env vars? i was getting 404's on the curl for MTX. After I got that build I am seeing this

[WyzeBridge] [+] Adding Living Room [HL_PAN3]
[WyzeBridge] starting MediaMTX 0.23.6
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/app/wyze_bridge.py", line 35, in run
    self.rtsp.start()
  File "/app/wyzebridge/rtsp_server.py", line 83, in start
    self.sub_process = Popen(["/app/mediamtx", "/app/mediamtx.yml"])
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.11/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: '/app/mediamtx'

[mrlt8]

There were some changes that moves the default ENVs. Can you try to build with v2.3.5 or the dockerfiles branch?

version: '2.4'
services:
    wyze-bridge:
        container_name: wyze-bridge
        restart: unless-stopped
        ports:
            - 8554:8554 # RTSP
            - 5000:5000 # WEB-UI
        build:
            context: ./app
            dockerfile: Dockerfile.qsv
        devices:
            - /dev/:/dev/
        environment:
            - WYZE_EMAIL=
            - WYZE_PASSWORD=
            - FFMPEG_CMD=

[mitchross]

There were some changes that moves the default ENVs. Can you try to build with v2.3.5 or the dockerfiles branch?

version: '2.4'
services:
    wyze-bridge:
        container_name: wyze-bridge
        restart: unless-stopped
        ports:
            - 8554:8554 # RTSP
            - 5000:5000 # WEB-UI
        build:
            context: ./app
            dockerfile: Dockerfile.qsv
        devices:
            - /dev/:/dev/
        environment:
            - WYZE_EMAIL=
            - WYZE_PASSWORD=
            - FFMPEG_CMD=

Yea my last messages were built on the 2.3.5 branch, not my own... The last error in the my last post was based on the dockerfiles branch.. I see you made a commit so ill try again tonight!

[mrlt8]

I believe you'll also need to set the ENV LIBVA_DRIVER_NAME= to i965 or iHD.

[mitchross]

When I build/run the compose file I get

[+] Building 6.8s (9/11)
 => [internal] load .dockerignore                                                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                                                 0.0s
 => [internal] load build definition from Dockerfile.qsv                                                                                                                        0.0s
 => => transferring dockerfile: 1.95kB                                                                                                                                          0.0s
 => [internal] load metadata for docker.io/amd64/python:3.11-slim-bullseye                                                                                                      0.4s
 => [internal] load build context                                                                                                                                               0.0s
 => => transferring context: 2.17kB                                                                                                                                             0.0s
 => [base 1/1] FROM docker.io/amd64/python:3.11-slim-bullseye@sha256:ed0e7c504b2b8c4517f703252811a6d2ddff5c9320272103d8cc72befbf950a9                                           0.0s
 => CACHED [builder 1/4] RUN echo 'deb http://deb.debian.org/debian testing main non-free' >/etc/apt/sources.list.d/debian-testing.list &&    apt-get update &&    apt-get ins  0.0s
 => CACHED [builder 2/4] COPY . /build/app/                                                                                                                                     0.0s
 => CACHED [builder 3/4] RUN pip3 install --disable-pip-version-check --prefix=/build/usr/local -r /build/app/requirements.txt                                                  0.0s
 => ERROR [builder 4/4] RUN cd /build && . app/.env &&    mkdir -p tokens img usr/lib/x86_64-linux-gnu/dri &&    curl -L https://github.com/BtbN/FFmpeg-Builds/releases/downlo  6.4s
------
 > [builder 4/4] RUN cd /build && . app/.env &&    mkdir -p tokens img usr/lib/x86_64-linux-gnu/dri &&    curl -L https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz     | tar --strip-components=1 -C usr/local -Jxf - --wildcards '*ffmpeg' &&    curl -L https://github.com/bluenviron/mediamtx/releases/download/v${MTX_TAG}/mediamtx_v${MTX_TAG}_linux_amd64.tar.gz     | tar xzf - -C app &&    cp app/amd.lib usr/local/lib/libIOTCAPIs_ALL.so &&    cp -R /usr/lib/x86_64-linux-gnu/dri/ usr/lib/x86_64-linux-gnu/dri/ &&    rm app/*.txt app/*.lib app/*.md:
#0 0.286   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
#0 0.286                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 96.4M  100 96.4M    0     0  16.0M      0  0:00:05  0:00:05 --:--:-- 16.7M
#0 6.296 curl: (3) URL using bad/illegal format or missing URL
#0 6.297
#0 6.297 gzip: stdin: unexpected end of file
#0 6.297 tar: Child returned status 1
#0 6.297 tar: Error is not recoverable: exiting now
------
failed to solve: process "/bin/sh -c cd /build && . app/.env &&    mkdir -p tokens img usr/lib/x86_64-linux-gnu/dri &&    curl -L https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz     | tar --strip-components=1 -C usr/local -Jxf - --wildcards '*ffmpeg' &&    curl -L https://github.com/bluenviron/mediamtx/releases/download/v${MTX_TAG}/mediamtx_v${MTX_TAG}_linux_amd64.tar.gz     | tar xzf - -C app &&    cp app/amd.lib usr/local/lib/libIOTCAPIs_ALL.so &&    cp -R /usr/lib/x86_64-linux-gnu/dri/ usr/lib/x86_64-linux-gnu/dri/ &&    rm app/*.txt app/*.lib app/*.md" did not complete successfully: exit code: 2
PS C:\Users\thell\Documents\Programming\docker-wyze-bridge> 

[mrlt8]

hmm, I wonder if it's not reading the MTX_TAG from the .env file for some reason..?

btw, the Dockerfile.qsv file should now be in the main branch: https://github.com/mrlt8/docker-wyze-bridge/blob/main/app/Dockerfile.qsv

[mrlt8]

Prebuilt images with the intel drivers should be available on dockerhub or ghcr.io using the latest-qsv tag

[392media]

Running the prebuilt image pulled with :latest-qsv, got this error: implib-gen: libX11.so.6: failed to load library 'libX11.so.6' via dlopen: libX11.so.6: cannot open shared object file: No such file or directory ffmpeg: libX11.so.6.init.c:61: load_library: Assertion `0 && "Assertion in generated code"' failed.

App version shown in the log is 2.3.7

[mitchross]

hmm, I wonder if it's not reading the MTX_TAG from the .env file for some reason..?

btw, the Dockerfile.qsv file should now be in the main branch: https://github.com/mrlt8/docker-wyze-bridge/blob/main/app/Dockerfile.qsv

Docker compose needs the .env file at the same level as the docker-compose.yml file. This means you'd need to move it to root level or move compose to the app folder...

Next the intel drives need to be part of the final image

https://github.com/mrlt8/docker-wyze-bridge/blob/0bb52a992ff61d3f12cd2db73007c67f607fbad8/app/Dockerfile.qsv#L9

When you are installing intel-media-driver it should be in the last FROM otherwise its dismissed

[mrlt8]

@392media could you try the latest image?

@mitchross I'm able to build with docker-compose using:

 build:
      context: ./app
      dockerfile: Dockerfile.qsv

or docker build -t wbqsv -f app/Dockerfile.qsv app

[mitchross]

@392media could you try the latest image?

@mitchross I'm able to build with docker-compose using:

 build:
      context: ./app
      dockerfile: Dockerfile.qsv

or docker build -t wbqsv -f app/Dockerfile.qsv app

got it... you still need to move the intel drives down to the last FROM

RUN echo 'deb http://deb.debian.org/debian testing main non-free' >/etc/apt/sources.list.d/debian-testing.list \ && apt-get -qq update \ && apt-get -qq install --no-install-recommends --no-install-suggests -y \ intel-opencl-icd \ mesa-va-drivers libva-drm2 intel-media-va-driver-non-free i965-va-driver libmfx1 radeontop intel-gpu-tools \ && apt-get -qq install --no-install-recommends --no-install-suggests -y \ i965-va-driver-shaders \ && rm -f /etc/apt/sources.list.d/debian-testing.list

[mrlt8]

I believe they should be in the final image:

[392media]

Tested app version 2.3.8 from prebuilt image. got this error

implib-gen: libva-drm.so.2: failed to load library 'libva-drm.so.2' via dlopen: libva-drm.so.2: cannot open shared object file: No such file or directory ffmpeg: libva-drm.so.2.init.c:61: load_library: Assertion `0 && "Assertion in generated code"' failed.

[mrlt8]

Added libva-drm2 to the edge build: wyze-bridge:edge-qsv

Please let me know if we're missing anything else.

[mitchross]

Added libva-drm2 to the edge build: wyze-bridge:edge-qsv

Please let me know if we're missing anything else.

Im pretty sure you will need all of these

RUN echo 'deb http://deb.debian.org/debian testing main non-free' >/etc/apt/sources.list.d/debian-testing.list \
    && apt-get -qq update \
    && apt-get -qq install --no-install-recommends --no-install-suggests -y \
        intel-opencl-icd \
        mesa-va-drivers libva-drm2 intel-media-va-driver-non-free i965-va-driver libmfx1 radeontop intel-gpu-tools \
    && apt-get -qq install --no-install-recommends --no-install-suggests -y \
        i965-va-driver-shaders \
    && rm -f /etc/apt/sources.list.d/debian-testing.list

https://gist.github.com/mitchross/5dc213dc6d4165704f379be414044235

[mitchross]

Tested app version 2.3.8 from prebuilt image. got this error

implib-gen: libva-drm.so.2: failed to load library 'libva-drm.so.2' via dlopen: libva-drm.so.2: cannot open shared object file: No such file or directory ffmpeg: libva-drm.so.2.init.c:61: load_library: Assertion `0 && "Assertion in generated code"' failed.

What do you have as your ffmpeg command ?

also @mrlt8 for qsv , what do you expect folks to pass in the ffmeg args? Same as what you posted earlier?

[392media]

the :edeg-qsv image works fine.

my use of it is to rotate the doorbell cam's image

[mitchross]

the :edeg-qsv image works fine.

my use of it is to rotate the doorbell cam's image

If you run "intel_gpu_top" on your box that's hosting this wyze bridge, is ffmepg active? You will need to install it with apt-get install intel-gpu-tools

[mitchross]

I am running edge-latest

CPU is still spiked

GPU its not being used

When I run frigate's ffmpeg ( docker gist i provided above ) I see

[392media]

@mrlt8 is it possible to prebuilt the ffmpeg command for doorbell image rotation with GPU acceleration? so it can be turned on/off with an ENV variable?

the key part of my ffmpeg line is -vf format=vaapi|nv12,hwupload,transpose_vaapi=1

[mrlt8]

@392media Can you share the ffmpeg command that you're using? dev-qsv should add the video filter config if the ENV H264_ENC=h264_qsv, but may need some further tweaking.

[mitchross]

Doorbell cam doesnt seem to rotate on dev-qsv

- name: ROTATE_DOOR
  value: "True"
- name: FRESH_DATA
  value: "True"
- name: H264_ENC
  value: h264_qsv
- name: LIBVA_DRIVER_NAME
  value: i965

[mrlt8]

Hmm anything in the logs?

[mitchross]

Hmm anything in the logs?

[WyzeBridge] 🎉 Connecting to WyzeCam Doorbell - Wyze Doorbell on 192.168.1.104
[WyzeBridge] 🎉 Connecting to WyzeCam Doorbell - Front door on 192.168.1.112
[wyze-doorbell] ☁️ Camera is connected via RELAY mode!!
[wyze-doorbell] Stream may consume additional bandwidth!
[wyze-doorbell] 📡 Getting 180kb/s HD stream (H264/20fps) via RELAY mode (WiFi: 95%) FW: 4.25.1.314 🔒 (DTLS) (2/3)
[wyze-doorbell] Re-encoding using h264_qsv [transpose='clock']
[wyze-doorbell] 📹 Will record 1800s mp4 clips to /record/wyze-doorbell/
Error parsing global options: Input/output error
[wyze-doorbell] WARNING: Skipping smaller frame at start of stream (frame_size=4)
[front-door] ☁️ Camera is connected via RELAY mode!!
[front-door] Stream may consume additional bandwidth!
[front-door] 📡 Getting 180kb/s HD stream (H264/20fps) via RELAY mode (WiFi: 85%) FW: 4.25.1.314 🔒 (DTLS) (2/3)
[front-door] Re-encoding using h264_qsv [transpose='clock']
[front-door] 📹 Will record 1800s mp4 clips to /record/front-door/
Error parsing global options: Input/output error
[front-door] WARNING: Skipping smaller frame at start of stream (frame_size=4)
[front-door] FFMPEG stopped

[mrlt8]

@mitchross could you try:

- FFMPEG_CMD=ffmpeg -loglevel info -f h264 -i - -c:v h264_qsv -filter:v transpose=clock -profile:v high -preset fast -force_key_frames expr:gte(t,n_forced*2) -movflags +empty_moov+default_base_moof+frag_keyframe -f rtsp -rtsp_transport tcp rtsp://0.0.0.0:8554/{cam_name}

[mitchross]

ffmpeg -loglevel info -f h264 -i - -c:v h264_qsv -filter:v transpose=clock -profile:v high -preset fast -force_key_frames expr:gte(t,n_forced*2) -movflags +empty_moov+default_base_moof+frag_keyframe -f rtsp -rtsp_transport tcp rtsp://0.0.0.0:8554/{cam_name}

No luck

[WyzeBridge] 🎉 Connecting to WyzeCam V3 - Shed on 192.168.1.185
[deck] FFMPEG stopped
Input #0, h264, from 'fd:':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 1920x1080, 25 fps, 40 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv))
[h264_qsv @ 0x557a18c44f80] Error creating a MFX session: -9.
[vost#0:0/h264_qsv @ 0x557a18c31640] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
Conversion failed!
[kitchen] ☁️ Camera is connected via RELAY mode!!
[kitchen] Stream may consume additional bandwidth!
[kitchen] 📡 Getting 180kb/s HD stream (H264/20fps) via RELAY mode (WiFi: 88%) FW: 4.36.11.4679 🔒 (DTLS) (2/3)
ffmpeg version N-111314-gb40324d417-20230701 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.1.0 (crosstool-NG 1.25.0.196_227d99d)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20230701
  libavutil      58. 13.101 / 58. 13.101
  libavcodec     60. 22.100 / 60. 22.100
  libavformat    60. 10.100 / 60. 10.100
  libavdevice    60.  2.100 / 60.  2.100
  libavfilter     9.  8.102 /  9.  8.102
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
[kitchen] WARNING: Skipping smaller frame at start of stream (frame_size=1)
[front-door] ☁️ Camera is connected via RELAY mode!!
[front-door] Stream may consume additional bandwidth!
[front-door] 📡 Getting 180kb/s HD stream (H264/20fps) via RELAY mode (WiFi: 83%) FW: 4.25.1.314 🔒 (DTLS) (2/3)
ffmpeg version N-111314-gb40324d417-20230701 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.1.0 (crosstool-NG 1.25.0.196_227d99d)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20230701
  libavutil      58. 13.101 / 58. 13.101
  libavcodec     60. 22.100 / 60. 22.100
  libavformat    60. 10.100 / 60. 10.100
  libavdevice    60.  2.100 / 60.  2.100
  libavfilter     9.  8.102 /  9.  8.102
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
[dog-cam-2] FFMPEG stopped
[front-door] WARNING: Skipping smaller frame at start of stream (frame_size=4)
Input #0, h264, from 'fd:':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 1920x1080, 25 fps, 40 tbr, 1200k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv))
[h264_qsv @ 0x5556ff9d6140] Error creating a MFX session: -9.
[vost#0:0/h264_qsv @ 0x5556ff9d6f80] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
Conversion failed!
[living-room] ☁️ Camera is connected via RELAY mode!!
[living-room] Stream may consume additional bandwidth!
[living-room] 📡 Getting 180kb/s HD stream (H264/20fps) via RELAY mode (WiFi: 70%) FW: 4.50.3.4240 🔒 (DTLS) (2/3)
ffmpeg version N-111314-gb40324d417-20230701 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.1.0 (crosstool-NG 1.25.0.196_227d99d)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20230701
  libavutil      58. 13.101 / 58. 13.101
  libavcodec     60. 22.100 / 60. 22.100
  libavformat    60. 10.100 / 60. 10.100
  libavdevice    60.  2.100 / 60.  2.100
  libavfilter     9.  8.102 /  9.  8.102
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
[living-room] WARNING: Skipping smaller frame at start of stream (frame_size=1)
[garage] FFMPEG stopped

[mrlt8]

Could you try this?

- FFMPEG_CMD=ffmpeg -loglevel info -f h264 -c:v h264_qsv -i - -c:v h264_qsv -filter:v format=nv12,hwupload,transpose=clock -profile:v high -preset fast -force_key_frames expr:gte(t,n_forced*2) -movflags +empty_moov+default_base_moof+frag_keyframe -f rtsp -rtsp_transport tcp rtsp://0.0.0.0:8554/{cam_name}

or this with hw decoding:

- FFMPEG_CMD=ffmpeg -loglevel info -hwaccel qsv -qsv_device /dev/dri/renderD128 -f h264 -c:v h264_qsv -i - -c:v h264_qsv -filter:v format=nv12,hwupload,transpose=clock -profile:v high -preset fast -force_key_frames expr:gte(t,n_forced*2) -movflags +empty_moov+default_base_moof+frag_keyframe -f rtsp -rtsp_transport tcp rtsp://0.0.0.0:8554/{cam_name}

sorry but I don't have any Intel GPUs so can't do any debugging myself.

[chris001]

More info here: https://trac.ffmpeg.org/wiki/Hardware/QuickSync

[mrlt8]

I can make changes to the container/ffmpeg if anyone has a working FFMPEG_CMD.

[392media]

I am using the edge:qsv image. My CPU is a kabylake i7. I am not using the _qsv variant but the _vaapi. The following ffmpeg line works for me.

ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -i - -vf transpose_vaapi=1 -c:v h264_vaapi -bf 0 -compression_level 3 -b:v 800k -rtsp_transport tcp -f rtsp rtsp://0.0.0.0:8554/{cam_name} My intention is to accelerate the whole rotation process, the filter "transpose=clock" is a CPU-based filter so is not suitable in my use case.

[392media]

It seems some libraries needs to be at a newer version to use QSV:

ffmpeg -loglevel verbose -hwaccel qsv -hwaccel_output_format qsv -c:v h264_qsv -i - -vf vpp_qsv=transpose=1 -c:v h264_qsv -preset fast -rtsp_transport tcp -f rtsp rtsp://0.0.0.0:8554/{cam_name} [h264 @ 0x55ea20e67500] Reinit context to 1728x1296, pix_fmt: yuv420p Input #0, h264, from 'fd:': Duration: N/A, bitrate: N/A Stream #0:0: Video: h264 (Main), 1 reference frame, yuv420p(tv, unknown/bt709/unknown, progressive, left), 1728x1296, 25 fps, 40 tbr, 1200k tbn [out#0/rtsp @ 0x55ea20fa92c0] No explicit maps, mapping streams automatically... [vost#0:0/h264_qsv @ 0x55ea20e8ef40] Created video stream from input stream 0:0 [AVHWDeviceContext @ 0x55ea20e87fc0] Trying to use DRM render node for device 0, with matching kernel driver (i915). [AVHWDeviceContext @ 0x55ea20e87fc0] libva: VA-API version 1.10.0 [AVHWDeviceContext @ 0x55ea20e87fc0] libva: User requested driver 'iHD' [AVHWDeviceContext @ 0x55ea20e87fc0] libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so [AVHWDeviceContext @ 0x55ea20e87fc0] libva: Found init function __vaDriverInit_1_10 [AVHWDeviceContext @ 0x55ea20e87fc0] libva: va_openDriver() returns 0 [AVHWDeviceContext @ 0x55ea20e87fc0] Initialised VAAPI connection: version 1.10 [AVHWDeviceContext @ 0x55ea20e87fc0] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 21.1.1 (). [AVHWDeviceContext @ 0x55ea20e87fc0] Driver not found in known nonstandard list, using standard behaviour. [AVHWDeviceContext @ 0x55ea20e87c00] Use Intel(R) oneVPL to create MFX session, API version is 2.9, the required implementation version is 1.3 [AVHWDeviceContext @ 0x55ea20e87c00] libva: Failed to get device id from the driver. Please consider to upgrade the driver to support VA-API 1.15.0 Device creation failed: -1313558101. No device available for decoder: device type qsv needed for codec h264_qsv. [vist#0:0/h264 @ 0x55ea20e90240] Hardware device setup failed for decoder: Unknown error occurred [vost#0:0/h264_qsv @ 0x55ea20e8ef40] Error initializing a simple filtergraph [AVIOContext @ 0x55ea20e76d80] Statistics: 382097 bytes read, 0 seeks [front-door] FFMPEG stopped

[mrlt8]

Thanks for the feedback @392media! Will try to see if I can figure out what's going on with the drivers.

Could you try the dev-qsv image with the env variable H264_ENC=h264_vaapi.

[mitchross]

Look at frigates docker files for driver reference. You might need Ubuntu as your base image On Tue, Jul 4, 2023 at 4:26 PM mrlt8 @.***> wrote:

Thanks for the feedback @392media https://github.com/392media! Will try to see if I can figure out what's going on with the drivers.

Could you try the dev-qsv image with the env variable H264_ENC=h264_vaapi.

— Reply to this email directly, view it on GitHub https://github.com/mrlt8/docker-wyze-bridge/issues/736#issuecomment-1620443880, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQJRCVX5GVHY4TDWRVMRYLXOQYY5ANCNFSM6AAAAAAVV3SHUU . You are receiving this because you were mentioned.Message ID: @.***>

[392media]

H264_ENC Not working.

[front-door] [FFMPEG_CMD] ffmpeg -loglevel verbose -fflags +genpts+flush_packets+nobuffer+bitexact -flags +low_delay -thread_queue_size 64 -analyzeduration 50 -probesize 50 -f H264 -i pipe: -flags +global_header -c:v h264_vaapi -filter:v transpose=clock -b:v 3000k -coder 1 -bufsize 1000k -profile:v main -preset ultrafast -forced-idr 1 -force_key_frames expr:gte(t,n_forced*2) -movflags +empty_moov+default_base_moof+frag_keyframe -muxdelay 0 -muxpreload 0 -map 0:v -f tee [f=rtsp:rtsp_transport=tcp:bsfs/v=dump_extra=freq=keyframe]rtsp://0.0.0.0:8554/front-door

For purpose of offload the doorbell rotation task, the decoder/rotate/encoder should all use vaapi or qsv.

[mrlt8]

hmm, seems like the dev-qsv image failed to build for some reason. Could you try again once the latest image gets pushed?

Also swapped fromi965-va-driver to i965-va-driver-shaders. So please let me know if that helps.

edit: latest dev-qsv image should be up now.

[392media]

the dev-qsv with H264_ENC=h264_vaapi is now using a new ffmpeg line:

[front-door] [FFMPEG_CMD] ffmpeg -loglevel verbose -fflags +genpts+flush_packets+nobuffer+bitexact -flags +low_delay -thread_queue_size 64 -hwaccel vaapi -hwaccel_output_format vaapi -analyzeduration 50 -probesize 50 -f H264 -i pipe: -flags +global_header -c:v h264_vaapi -filter:v transpose_vaapi=clock -b:v 2000k -coder 1 -bufsize 2000k -profile:v main -preset ultrafast -forced-idr 1 -force_key_frames expr:gte(t,n_forced*2) -movflags +empty_moov+default_base_moof+frag_keyframe -muxdelay 0 -muxpreload 0 -map 0:v -f tee [f=rtsp:rtsp_transport=tcp:bsfs/v=dump_extra=freq=keyframe]rtsp://0.0.0.0:8554/front-door

debug info from the encoder:

Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi)) [h264 @ 0x562e923a2580] Reinit context to 1728x1296, pix_fmt: vaapi [graph 0 input from stream 0:0 @ 0x562e92508280] w:1728 h:1296 pixfmt:vaapi tb:1/1200000 fr:20/1 sar:0/1 [h264_vaapi @ 0x562e92398980] Using input frames context (format vaapi) with h264_vaapi encoder. [h264_vaapi @ 0x562e92398980] Input surface format is nv12. [h264_vaapi @ 0x562e92398980] Using VAAPI profile VAProfileH264Main (6). [h264_vaapi @ 0x562e92398980] Using VAAPI entrypoint VAEntrypointEncSlice (6). [h264_vaapi @ 0x562e92398980] Using VAAPI render target format YUV420 (0x1). [h264_vaapi @ 0x562e92398980] Buffering settings are ignored in AVBR RC mode. [h264_vaapi @ 0x562e92398980] RC mode: AVBR. [h264_vaapi @ 0x562e92398980] RC target: 2000000 bps converging in 120 frames with 100% accuracy. [h264_vaapi @ 0x562e92398980] RC framerate: 20/1 (20.00 fps). [h264_vaapi @ 0x562e92398980] Driver does not report any additional prediction constraints. [h264_vaapi @ 0x562e92398980] Using intra, P- and B-frames (supported references: 8 / 2). [h264_vaapi @ 0x562e92398980] All wanted packed headers available (wanted 0xd, found 0x1f).

Video is live but it's very laggy. The bitrate/speed might not be tuned well for livestreaming.

Frigate doesn't like the RTSP feed. Disconnecting every few seconds.

The stream produced by my customized ffmpeg line with edge-qsv image works well in Frigate.

The VA-API version is still not allow _qsv coder to run. [AVHWDeviceContext @ 0x5562e6d33b80] libva: VA-API version 1.10.0

[chris001]

I got this command to work, for an Intel iGPU, using its built-in h264_vaapi (the legacy pre-"Broadwell" hardware accelerator):

ffmpeg -loglevel info -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i "mysource.mp4" -vf 'format=nv12,hwupload' -threads 8 -y -f mp4 -acodec copy -b:v 3000k -vcodec h264_vaapi "output.mp4"

Parameters could be tweaked further.

Using image: wyze-bridge:dev-qsv

EDIT: Before running the ffmpeg command, install the intel hardware acceleration drivers inside the wyze-bridge container:

apt -qq update -y && apt install -qq sudo wget apt-utils -y
echo "Install Intel video hardware acceleration packages"
sudo echo 'deb http://deb.debian.org/debian testing main non-free' >/etc/apt/sources.list.d/debian-testing.list \
  && sudo apt-get -qq update -y  \
  && sudo apt-get install --no-install-recommends --no-install-suggests -y  \
         intel-opencl-icd vainfo         mesa-va-drivers libva-drm2 intel-media-va-driver-non-free \
         i965-va-driver libmfx1 radeontop intel-gpu-tools  \
  && sudo apt-get install --no-install-recommends --no-install-suggests -y  \
         i965-va-driver-shaders  \
  && sudo rm -f /etc/apt/sources.list.d/debian-testing.list

The output is:

ffmpeg version N-111327-g50f34172e0-20230703 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13.1.0 (crosstool-NG 1.25.0.196_227d99d)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20230703
  libavutil      58. 13.101 / 58. 13.101
  libavcodec     60. 22.100 / 60. 22.100
  libavformat    60. 10.100 / 60. 10.100
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9.  8.102 /  9.  8.102
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
[AVHWDeviceContext @ 0x560c06163e80] libva: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'mysource.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41isomavc1
    creation_time   : 2020-11-17T09:28:51.000000Z
  Duration: 00:00:05.87, start: 0.000000, bitrate: 5272 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 5016 kb/s, 50 fps, 50 tbr, 50 tbn (default)
    Metadata:
      creation_time   : 2020-11-17T09:28:51.000000Z
      handler_name    : L-SMASH Video Handler
      vendor_id       : [0][0][0][0]
      encoder         : AVC Coding
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 253 kb/s (default)
    Metadata:
      creation_time   : 2020-11-17T09:28:51.000000Z
      handler_name    : L-SMASH Audio Handler
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Output #0, mp4, to 'output.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41isomavc1
    encoder         : Lavf60.10.100
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), vaapi(tv, bt709, progressive), 1920x1080, q=2-31, 3000 kb/s, 50 fps, 12800 tbn (default)
    Metadata:
      creation_time   : 2020-11-17T09:28:51.000000Z
      handler_name    : L-SMASH Video Handler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.22.100 h264_vaapi
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 253 kb/s (default)
    Metadata:
      creation_time   : 2020-11-17T09:28:51.000000Z
      handler_name    : L-SMASH Audio Handler
      vendor_id       : [0][0][0][0]
[out#0/mp4 @ 0x560c061bb440] video:2072kB audio:181kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.345502%
frame=  293 fps= 27 q=-0.0 Lsize=    2261kB time=00:00:05.84 bitrate=3168.8kbits/s speed=0.535x

[mitchross]

I got this command to work, for an Intel iGPU, using its built-in h264_vaapi (the legacy pre-"Broadwell" hardware accelerator):

ffmpeg -loglevel info -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -i "mysource.mp4" -vf 'format=nv12,hwupload' -threads 8 -y -f mp4 -acodec copy -b:v 3000k -vcodec h264_vaapi "output.mp4"

Parameters could be tweaked further.

Using image: wyze-bridge:dev-qsv

EDIT: Before running the ffmpeg command, install the intel hardware acceleration drivers inside the wyze-bridge container:

apt -qq update -y && apt install -qq sudo wget apt-utils -y

echo "Install Intel video hardware acceleration packages"

sudo echo 'deb http://deb.debian.org/debian testing main non-free' >/etc/apt/sources.list.d/debian-testing.list \

  && sudo apt-get -qq update -y  \

  && sudo apt-get install --no-install-recommends --no-install-suggests -y  \

         intel-opencl-icd vainfo         mesa-va-drivers libva-drm2 intel-media-va-driver-non-free \

         i965-va-driver libmfx1 radeontop intel-gpu-tools  \

  && sudo apt-get install --no-install-recommends --no-install-suggests -y  \

         i965-va-driver-shaders  \

  && sudo rm -f /etc/apt/sources.list.d/debian-testing.list

The output is:

ffmpeg version N-111327-g50f34172e0-20230703 Copyright (c) 2000-2023 the FFmpeg developers

  built with gcc 13.1.0 (crosstool-NG 1.25.0.196_227d99d)

  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20230703

  libavutil      58. 13.101 / 58. 13.101

  libavcodec     60. 22.100 / 60. 22.100

  libavformat    60. 10.100 / 60. 10.100

  libavdevice    60.  2.101 / 60.  2.101

  libavfilter     9.  8.102 /  9.  8.102

  libswscale      7.  3.100 /  7.  3.100

  libswresample   4. 11.100 /  4. 11.100

  libpostproc    57.  2.100 / 57.  2.100

[AVHWDeviceContext @ 0x560c06163e80] libva: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'mysource.mp4':

  Metadata:

    major_brand     : mp42

    minor_version   : 0

    compatible_brands: mp42mp41isomavc1

    creation_time   : 2020-11-17T09:28:51.000000Z

  Duration: 00:00:05.87, start: 0.000000, bitrate: 5272 kb/s

  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 5016 kb/s, 50 fps, 50 tbr, 50 tbn (default)

    Metadata:

      creation_time   : 2020-11-17T09:28:51.000000Z

      handler_name    : L-SMASH Video Handler

      vendor_id       : [0][0][0][0]

      encoder         : AVC Coding

  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 253 kb/s (default)

    Metadata:

      creation_time   : 2020-11-17T09:28:51.000000Z

      handler_name    : L-SMASH Audio Handler

      vendor_id       : [0][0][0][0]

Stream mapping:

  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))

  Stream #0:1 -> #0:1 (copy)

Press [q] to stop, [?] for help

Output #0, mp4, to 'output.mp4':

  Metadata:

    major_brand     : mp42

    minor_version   : 0

    compatible_brands: mp42mp41isomavc1

    encoder         : Lavf60.10.100

  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), vaapi(tv, bt709, progressive), 1920x1080, q=2-31, 3000 kb/s, 50 fps, 12800 tbn (default)

    Metadata:

      creation_time   : 2020-11-17T09:28:51.000000Z

      handler_name    : L-SMASH Video Handler

      vendor_id       : [0][0][0][0]

      encoder         : Lavc60.22.100 h264_vaapi

  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 253 kb/s (default)

    Metadata:

      creation_time   : 2020-11-17T09:28:51.000000Z

      handler_name    : L-SMASH Audio Handler

      vendor_id       : [0][0][0][0]

[out#0/mp4 @ 0x560c061bb440] video:2072kB audio:181kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.345502%

frame=  293 fps= 27 q=-0.0 Lsize=    2261kB time=00:00:05.84 bitrate=3168.8kbits/s speed=0.535x

Can you run "intel_gpu_top" either on the host or in the container and see if the gpu is activated? Only run Wyze bridge when doing this. Or you will get false positives if running frigate.