search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.55k stars 1.69k forks source link

[Support]: Cant get HW accel working on amdgpu R5 340 #2597

Closed Kosygor closed 2 years ago

Kosygor commented 2 years ago

Describe the problem you are having

I just cannot get hwaccel working. I am running Frigate NVR as addon for Home Asisstant supervised which is running inside Debian 11.1 VM on TrueNAS Scale. My card is Radeon R5 340. I am still waiting to receive my Coral TPU. With provided config I can see video of camera but CPU is high.

Version

0.9.4-26AE608

Frigate config file

environment_vars:
  LIBVA_DRIVER_NAME: radeonsi

detectors:
  cpu1:
    type: cpu
mqtt:
  host: 192.168.0.111
  user: usr
  password: pwd
cameras:
  Taras:
    ffmpeg:
      inputs:
        - path: rtsp://admin:admin@192.168.0.241:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
    detect:
      enabled: False
      # width: 2048
      # height: 1536
      # fps: 20
ffmpeg:      
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128

logger:
      # Optional: Default log verbosity (default: shown below)
  default: info
      # Optional: Component specific logger overrides
  logs:
    ffmpeg.taras.record: debug

Relevant log output

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2022-01-06 19:06:17] frigate.app                    INFO    : Starting Frigate (0.9.4-26ae608)
[2022-01-06 19:06:17] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2022-01-06 19:06:17] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2022-01-06 19:06:17] peewee_migrate                 INFO    : There is nothing to migrate
[2022-01-06 19:06:17] frigate.mqtt                   INFO    : MQTT connected
[2022-01-06 19:06:17] detector.cpu1                  INFO    : Starting detection process: 242
[2022-01-06 19:06:17] frigate.edgetpu                WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
[2022-01-06 19:06:17] frigate.app                    INFO    : Output process started: 244
[2022-01-06 19:06:17] ws4py                          INFO    : Using epoll
[2022-01-06 19:06:17] frigate.app                    INFO    : Camera processor started for Taras: 248
[2022-01-06 19:06:17] frigate.app                    INFO    : Capture process started for Taras: 250
[2022-01-06 19:06:17] ws4py                          INFO    : Using epoll

FFprobe output from your camera

root@ccab4aaf-frigate:/opt/frigate# ffmpeg rtsp://admin:admin@192.168.0.241:554/cam/realmonitor?channel=1&subtype=0
[1] 331
root@ccab4aaf-frigate:/opt/frigate# ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100

Frigate stats

{"Taras":{"camera_fps":5.0,"capture_pid":250,"detection_fps":0.0,"pid":248,"process_fps":0.0,"skipped_fps":0.0},"detection_fps":0.0,"detectors":{"cpu1":{"detection_start":0.0,"inference_speed":10.0,"pid":242}},"service":{"storage":{"/dev/shm":{"free":2054.7,"mount_type":"tmpfs","total":2056.4,"used":1.6},"/media/frigate/clips":{"free":80528.4,"mount_type":"ext4","total":131029.2,"used":43797.6},"/media/frigate/recordings":{"free":80528.4,"mount_type":"ext4","total":131029.2,"used":43797.6},"/tmp/cache":{"free":2056.4,"mount_type":"tmpfs","total":2056.4,"used":0.0}},"uptime":595,"version":"0.9.4-26ae608"}}

Operating system

HassOS

Install method

HassOS Addon

Coral version

CPU (no coral)

Network connection

Wired

Camera make and model

BCS-DMIP1200IR-E

Any other information that may be helpful

And here is the thing when I run above I get

root@ccab4aaf-frigate:/opt/frigate# vainfo
error: XDG_RUNTIME_DIR not set in the environment.
error: can't connect to X server!
libva info: VA-API version 1.12.0
libva info: User environment variable requested driver 'i965'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_6
DRM_IOCTL_I915_GEM_APERTURE failed: Invalid argument
Assuming 131072kB available aperture size.
May lead to reduced performance or incorrect rendering.
get chip id failed: -1 [2]
param: 4, val: 0
libva error: /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so init failed
libva info: va_openDriver() returns -1
vaInitialize failed with error code -1 (unknown libva error),exit

But when I run manually run inside container : LIBVA_DRIVER_NAME="radeonsi"

I get 

root@ccab4aaf-frigate:/opt/frigate# vainfo
error: XDG_RUNTIME_DIR not set in the environment.
error: can't connect to X server!
libva info: VA-API version 1.12.0
libva info: User environment variable requested driver 'radeonsi'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_7
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.12 (libva 2.12.0)
vainfo: Driver version: Mesa Gallium driver 21.2.4 - kisak-mesa PPA for AMD Radeon (TM) R5 340 (OLAND, DRM 3.40.0, 5.10.0-10-amd64, LLVM 12.0.1)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

But CPU usage is still high. When I comment out 

environment_vars:
  LIBVA_DRIVER_NAME: radeonsi

From above config restart frigate and then once again set env var manually: I got no image... ffmpeg result does not change but Log gives this:

[2022-01-06 19:26:29] frigate.video                  INFO    : Taras: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures
[2022-01-06 19:26:29] frigate.video                  INFO    : Taras: ffmpeg process is not running. exiting capture thread...

From my uderstaning for some reason frigate does not load this env_var at the beggining. I have tried to put this env var at the beggining and the end of config file.

Edit: ok I missed disabling protection mode. Can not test right now becasue disable option is gone from my HA WebUI.

Installed Firgate Full Access (reason that i did not protection mode button) turnet it off but nothing changed. 

sudo docker exec addon_ccab4aaf_frigate-fa env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=ccab4aaf-frigate-fa
CONFIG_FILE=/config/frigate.yml
TZ=Europe/Warsaw
SUPERVISOR_TOKEN=c54c661dba0b2d3542da9edd5ee1faadbf6cd6ce98ad279715c14c56ca9d5e57a0171c33e46e5ef4465321e6eeef8042afc7481b011bc5cf
HASSIO_TOKEN=c54c661dba0b2d3542da9edd5ee1faadbf6cd6ce98ad279715c14c56ca9d5e57a0171c33e46e5ef4465321e6eeef8042afc7481b011bc5cf
FLASK_ENV=development
DEBIAN_FRONTEND=noninteractive
LIBVA_DRIVER_NAME=i965
HOME=/root
blakeblackshear commented 2 years ago

The environment variables are only set in the python process and the ffmpeg processes it spawns. It won't be set when you exec into the container, but it is working. It doesn't matter where you set them in the config.

Is the resolution and frame rate commented out correct for the stream you are connecting to? Do you have a lower resolution substream you could use for detection? I think you are underestimating the CPU required to decode high resolutions even with acceleration.

Kosygor commented 2 years ago

The environment variables are only set in the python process and the ffmpeg processes it spawns. It won't be set when you exec into the container, but it is working. It doesn't matter where you set them in the config.

Right, noticed varabile behaviour after posting issue, but did not have time to describe it here.

Do you have a lower resolution substream you could use for detection?

Since my TPU is on the way I am only testing now for record with detect disabled.

Is the resolution and frame rate commented out correct for the stream you are connecting to? Resolution is diffrent now. But for some reason ffmpeg command in htop has 720p resolution mentioned. I think you are underestimating the CPU required to decode high resolutions even with acceleration. Propably yes but considering what my colegue noticed I will do some more test when TPU will arrive.


root       61319 11.7  1.2 361600 49552 ?        Ssl  23:03   1:02 ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:
root       61320  0.6  1.7 1722284 70624 ?       Sl   23:03   0:03 frigate.process:Taras
root       61321  2.3  1.7 1735896 69992 ?       Sl   23:03   0:12 python3 -u -m frigate
root       61322  0.0  0.3  74624 12100 ?        Ss   23:03   0:00 ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:

So all in all (for futere readers) I had few more things:

  1. You should always check if GPU is running in dedicated software (amd >radeontop) 1 FHD stream \/ image
  2. In my case if resolution was to high (3 Mpix) for GPU it somehow refused to hw accell... after decreasing it do FHD there was greater load on GPU as above)
  3. vainfo libva error inside or outside container is not reason to think there is sometihing to fix.
  4. My friend with similarar procesor ( in terms of power but 5 year newer J5005 so h.254 hwaccel intergrated and no GPU) with 4 x 2k and 2 x FHD cameras switched "off" detect, hwaccel and TPU and he says he had 15% CPU Load more compered to when he had enabled hwaccel and detect with TPU "on "So I am assuming ffmeg is somehow.
blakeblackshear commented 2 years ago

Since my TPU is on the way I am only testing now for record with detect disabled

Even with detect disabled, it still has to decode the stream to show the camera in the UI and for the APIs to work. You would still be better off with a lower resolution set for detect.

Kosygor commented 2 years ago

Ok do at this point it seams that it does not matter if hwaccel is enabled or not. If You allow I will report how situation has changed after installing TPU.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.