search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.59k stars 1.69k forks source link

[HW Accel Support]: ffmpeg still using heavy CPU loads #8613

Closed R3STEAS closed 9 months ago

R3STEAS commented 11 months ago

Bildschirmfoto 2023-11-14 um 16 10 04

Describe the problem you are having

I recently upgraded to an GTX 1050Ti and get through the hw accel guide. Unfortunately only 156mb per cam is processed by the GPU

Version

0.13.0-08588599

Frigate config file

mqtt: 
  enabled: True 
  host: 192.168.178.24 
  port: 21183 
  topic_prefix: frigate 
  client_id: frigate 
  user: mqtt 
  password: pw 
  stats_interval: 30

detectors: # <---- add detectors
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  hwaccel_args: preset-nvidia-h264

go2rtc:
  ffmpeg:
    hwaccel_args: preset-nvidia-h264
  streams:
    carport:
      - "ffmpeg:rtsp://xxxx:xxxx@192.168.178.107:554/live/ch0#video=h264#raw=-i \"/config/carport_overlay.png\" -filter_complex [0][1]overlay=x=(main_w-overlay_w)/2:y=(main_h-overlay_h)/2,drawtext=\"expansion=strftime:fontfile=/config/Cantarell-Regular.ttf:fontsize=42:fontcolor=white:shadowcolor=black:shadowx=2:shadowy=1:text='%d.%m.%Y\ %H\\:%M\\:%S':x=1830:y=8\""
    haustuer:
      - "ffmpeg:rtsp://xxxx:xxxx@192.168.178.109:554/live/ch0#video=h264#raw=-i \"/config/haustuer_overlay.png\" -filter_complex [0][1]overlay=x=(main_w-overlay_w)/2:y=(main_h-overlay_h)/2,drawtext=\"expansion=strftime:fontfile=/config/Cantarell-Regular.ttf:fontsize=42:fontcolor=white:shadowcolor=black:shadowx=2:shadowy=1:text='%d.%m.%Y\ %H\\:%M\\:%S':x=1830:y=8\""
cameras:
  haustuer:
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: rtsp://127.0.0.1:8554/haustuer
          input_args: preset-rtsp-restream
          roles:
            - detect
    objects:
      track:
        - person
        - car
        - bicycle
        - motorcycle
        - bus
        #- bird
        #- cat
        #- dog
    record:
      enabled: True
      retain:
        days: 0
        mode: motion
      events:
        retain:
          default: 14
          mode: active_objects
    snapshots:
      enabled: True
      clean_copy: False
    motion:
      mask:
        - 1605,40,2015,40,2015,0,1605,0

  carport:
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: rtsp://127.0.0.1:8554/carport
          input_args: preset-rtsp-restream
          roles:
            - detect
    objects:
      track:
        - person
        #- car
        #- bicycle
        #- motorcycle
        - bus
        - bird
        - cat
        - dog
    zones:
      bicycle:
        coordinates: 2050,450,2050,920,1495,920,1495,450
        objects:
            - person
            #- car
            - bus
            #- bird
            #- cat
            #- dog
        record:
          enabled: True
          retain:
            days: 0
            mode: motion
          events:
            objects:
              - person
              #- car
              - bus
              #- bird
              #- cat
              #- dog
            retain:
              default: 14
              mode: active_objects
      mudderpark:
        coordinates: 0,1080,0,745,0,410,1495,395,1495,1080
        objects:
            - person
            #- bicycle
            #- motorcycle
            - bus
            #- bird
            #- cat
            #- dog        
        record:
          enabled: True
          retain:
            days: 0
            mode: motion
          events:
            objects:
                - person
                #- bicycle
                #- motorcycle
                - bus
                #- bird
                #- cat
                #- dog 
            retain:
              default: 14
              mode: active_objects
    snapshots:
      enabled: True
      clean_copy: False
    motion:
      mask:
        - 1605,40,2015,40,2015,0,1605,0

detect:
  enabled: True
  width: 2048
  height: 1080
  fps: 15
  stationary:
    interval: 1
    threshold: 50     
record:
  enabled: True
  retain:
    days: 0
    mode: motion
  events:
    objects:
        - person
        #- bicycle
        #- car
        #- motorcycle
        - bus
        #- bird
        #- cat
        #- dog 
    retain:
      default: 14
      mode: active_objects

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: frigate:latest

    deploy:    # <------------- Add this section
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1 # number of GPUs
              capabilities: [gpu]

    shm_size: "128mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      #- /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/nvidia0:/dev/nvidia0 
      - /dev/nvidiactl:/dev/nvidiactl
      #- /dev/bus/usb/003/004:/dev/coral
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/raid/Docker/data/frigate/config:/config
      - /mnt/raid/Backups/camera/clips:/media/frigate/clips
      - /mnt/raid/Backups/camera/recordings:/media/frigate/recordings
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "pw"
    networks:
      - traefikproxy

networks:
  traefikproxy:
    external: true

Relevant log output

2023-11-14 15:53:56.031426905  [INFO] Preparing Frigate...
2023-11-14 15:53:56.104391201  [INFO] Starting Frigate...
2023-11-14 15:53:57.198002806  [2023-11-14 15:53:57] frigate.app                    INFO    : Starting Frigate (0.13.0-08588599)
2023-11-14 15:53:57.236134442  [2023-11-14 15:53:57] peewee_migrate.logs            INFO    : Starting migrations
2023-11-14 15:53:57.239388116  [2023-11-14 15:53:57] peewee_migrate.logs            INFO    : There is nothing to migrate
2023-11-14 15:53:57.250710351  [2023-11-14 15:53:57] frigate.app                    INFO    : Recording process started: 295
2023-11-14 15:53:57.253094410  [2023-11-14 15:53:57] frigate.app                    INFO    : go2rtc process pid: 101
2023-11-14 15:53:57.277265519  [2023-11-14 15:53:57] frigate.app                    INFO    : Output process started: 307
2023-11-14 15:53:57.295127468  [2023-11-14 15:53:57] frigate.app                    INFO    : Camera processor started for haustuer: 315
2023-11-14 15:53:57.300161188  [2023-11-14 15:53:57] frigate.app                    INFO    : Camera processor started for carport: 316
2023-11-14 15:53:57.305532627  [2023-11-14 15:53:57] frigate.app                    INFO    : Capture process started for haustuer: 318
2023-11-14 15:53:57.310968070  [2023-11-14 15:53:57] frigate.app                    INFO    : Capture process started for carport: 320
2023-11-14 15:54:00.492055792  [2023-11-14 15:53:57] detector.coral                 INFO    : Starting detection process: 305
2023-11-14 15:54:00.492061429  [2023-11-14 15:53:57] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2023-11-14 15:54:00.492067090  [2023-11-14 15:54:00] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

FFprobe output from your camera

root@fileserver:/mnt/raid/Docker/immich# ffprobe rtsp://cam1:cam1@192.168.178.107:554/live/ch0
ffprobe version 4.2.7-0ubuntu0.1+esm3 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1+esm3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[rtsp @ 0x55f76bec7f00] max delay reached. need to consume packet
[rtsp @ 0x55f76bec7f00] RTP: missed 17 packets
Input #0, rtsp, from 'rtsp://cam1:cam1@192.168.178.107:554/live/ch0':
  Metadata:
    title           : streamed by RTSP server
  Duration: N/A, start: 0.266000, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 2304x1296, 15 fps, 15 tbr, 90k tbn, 15 tbc
    Stream #0:1: Audio: aac, 8000 Hz, mono, fltp
root@fileserver:/mnt/raid/Docker/immich# ffprobe rtsp://cam2:cam2@192.168.178.109:554/live/ch0
ffprobe version 4.2.7-0ubuntu0.1+esm3 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1+esm3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, rtsp, from 'rtsp://cam2:cam2@192.168.178.109:554/live/ch0':
  Metadata:
    title           : streamed by RTSP server
  Duration: N/A, start: 0.069000, bitrate: N/A
    Stream #0:0: Video: hevc (Main), yuv420p(tv, bt709), 2304x1296, 15 fps, 15 tbr, 90k tbn, 15 tbc
    Stream #0:1: Audio: aac, 8000 Hz, mono, fltp

Operating system

Debian

Install method

Docker Compose

Network connection

Wired

Camera make and model

wansview 2k outdoor cam

Any other information that may be helpful

No response

NickM-27 commented 11 months ago

you are applying hwaccel to go2rtc incorrectly, preset-nvidia-h264 is a frigate preset and does not apply to go2rtc. You must follow go2rtc docs for enabling hwaccel for your transcoded streams https://github.com/AlexxIT/go2rtc/wiki/Hardware-acceleration

R3STEAS commented 11 months ago

Thanks for the reply! I understand that. But since I have to use #hardware, my #raw=-i ... overlay doesn't work anymore. Where should I place this now? I have to use a overlay because of some laws in my country, I am not able to stream public driveways etc.

NickM-27 commented 11 months ago

You'll need to adjust the raw -vf to handle passing between the hardware and software memory.

R3STEAS commented 11 months ago

Well now I don't understand anymore :D can you give me a hint to this please? :)

NickM-27 commented 11 months ago

I'm not familiar with -filter_complex so I'm not so sure what needs to change exactly, I only have experience with -vf

R3STEAS commented 11 months ago

Well if I can add an overlay png with -vf then I am willing to change :) I don't have much I want. Just HW accel and an overlay

github-actions[bot] commented 10 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.