search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
17.33k stars 1.6k forks source link

[HW Accel Support]: ffmpeg test with hardware acceleration on VIM3 causes segfault #4268

Closed tanglebridge closed 1 year ago

tanglebridge commented 1 year ago

Describe the problem you are having

I am trying to set up hardware acceleration. I have a VIM3/Amlogic A311D board (I use a coral USB TPU).

The board supports hardware acceleration for video and trying to make that work. I ran this line from documentation to test:

To test I run, inside the container: ffmpeg -c:v h264_v4l2m2m -re -stream_loop -1 -i https://streams.videolan.org/ffmpeg/incoming/720p60.mp4 -f rawvideo -pix_fmt yuv420p pipe: > /dev/null

Note the hardware acceleration arguments I added. In the docker compose file I added /dev/video0 to devices, because ffmpeg complained abount not being able to access it.

When I run the above command I get a segfault! See snippet below. This is an ffmpeg issue but I hope someone here has an idea. Either way it is a bug?

Thank you for making this!

# ffmpeg -c:v h264_v4l2m2m -re -stream_loop -1 -i https://streams.videolan.org/ffmpeg/incoming/720p60.mp4 -f rawvideo -pix_fmt yuv420p pipe: > /dev/null
ffmpeg version n5.1-2-g915ef932a3-20220731 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=aarch64-ffbuild-linux-gnu- --arch=aarch64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --disable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --disable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --disable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20220731
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://streams.videolan.org/ffmpeg/incoming/720p60.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.10.100
  Duration: 00:00:16.28, start: 0.000000, bitrate: 2503 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, smpte170m, progressive), 1280x720 [SAR 1:1 DAR 16:9], 2496 kb/s, 60 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
[h264_v4l2m2m @ 0xaaab1e353ef0] Using device /dev/video0
[h264_v4l2m2m @ 0xaaab1e353ef0] driver 'meson-vdec' on card 'Amlogic Video Decoder' in mplane mode
[h264_v4l2m2m @ 0xaaab1e353ef0] requesting formats: output=H264 capture=NM12
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_v4l2m2m) -> rawvideo (native))
Press [q] to stop, [?] for help
Segmentation fault (core dumped)

Version

0.11.1-2eada21

Frigate config file

mqtt:
  host: localhost

detectors:
  coral:
    type: edgetpu
    device: usb

snapshots:
  enabled: True

cameras:
  tapocam_1:
    ffmpeg:
      hwaccel_args: -c:v h264_v4l2m2m
      inputs:
        - path: rtsp://tpcam1:5R7mPGMb7@192.168.8.190:554/stream1
          roles:
            - detect
            - rtmp
            - record
    rtmp:
      enabled: True
    detect:
      width: 1920
      height: 1080
    record:
      enabled: True

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:stable
    shm_size: "64mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
        #- /dev/dri/renderD128:/dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
      - /dev/video0:/dev/video0
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/docker/frigate/config.yml:/config/config.yml:ro
      - /media/frigate:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    network_mode: "host"
      #ports:
      #- "5000:5000"
      #- "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: "password"

Relevant log output

NA

FFprobe output from your camera

$ ffprobe rtsp://tpcam1:5R7mPGMb7@192.168.8.190:554/stream1                                        
ffprobe version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2007-2021 the FFmpeg developers                            
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)                                                                   
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared                                                                                              
  libavutil      56. 70.100 / 56. 70.100                                                                        
  libavcodec     58.134.100 / 58.134.100                                                                        
  libavformat    58. 76.100 / 58. 76.100                                                                        
  libavdevice    58. 13.100 / 58. 13.100                                                                        
  libavfilter     7.110.100 /  7.110.100                                                                        
  libswscale      5.  9.100 /  5.  9.100                                                                        
  libswresample   3.  9.100 /  3.  9.100                                                                        
  libpostproc    55.  9.100 / 55.  9.100                                                                        
Input #0, rtsp, from 'rtsp://tpcam1:5R7mPGMb7@192.168.8.190:554/stream1':                                       
  Metadata:                                                                                                     
    title           : Session streamed by "TP-LINK RTSP Server"                                                 
    comment         : stream1                                                                                   
  Duration: N/A, start: 0.000000, bitrate: N/A                                                                  
  Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1920x1080, 15 fps, 15 tbr, 90k tbn, 30 tbc 
  Stream #0:1: Audio: pcm_alaw, 8000 Hz, 1 channels, s16, 64 kb/s

Operating system

Other Linux

Install method

Docker Compose

Network connection

Mixed

Camera make and model

d-link tapo

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

It's definitely a bug but not clear where, could also be a bug in the amlogic driver. Does this work if you run it on the host?

tanglebridge commented 1 year ago

Ah. My host has different hardware, and the point is to accelerate the hardware on the target. As you can see ffmpeg correctly loads the module. Might even be a driver issue - you are correct.

NickM-27 commented 1 year ago

By host I mean the os running docker. Instead of running ffmpeg in frigate (in the docker container) I'm asking if the results are different running it in the host OS.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.