search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.39k stars 1.78k forks source link

[HW Accel Support]: #5467

Closed mikerasmussen1 closed 1 year ago

mikerasmussen1 commented 1 year ago

Describe the problem you are having

Trying to decode and encode camera streams with NVIDIA GPU to offload CPU.

First of all, I have seen a few different things on this. Is HW accelerate encoding supported?

Went through a lot of pain to get the NVIDIA drivers up and running and working in Docker, I verified it works for decoding, but when I tried adding the necessary arguments for encoding with GPU I cannot get it to work,

Version

011.0

Frigate config file

mqtt:
  host: 192.168.86.52
  port: 1883
  user: frigate
  password: XXXXXXX
detectors:
  coral:
    type: edgetpu
    device: usb
    ffmpeg:
cameras:
    NewFront:
        ffmpeg:
            hwaccel_args: -hwaccel cuda -hwaccel_output_format cuda  #-hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
            output_args: 
                record:  -f segment -segment_time 10 -segment_format mp4 -strftime 1 -c:a copy -c:v h264_nvenc 
            inputs:
                - path: rtsp://admin:deadbeef11@192.168.86.94:554/cam/realmonitor?channel=1&subtype=1
        record:
            enabled: True
            retain:
                days: 60
                mode: all
            events:
                retain:
                    default: 180
                    mode: active_objects
        objects:
            track:
                - person
                - car
                - truck
                - cat
                - dog
timestamp_style:
  # Optional: Position of the timestamp (default: shown below)
  #           "tl" (top left), "tr" (top right), "bl" (bottom left), "br" (bottom right)
  position: "br"
  # Optional: Format specifier conform to the Python package "datetime" (default: shown below)
  #           Additional Examples:
  #             german: "%d.%m.%Y %H:%M:%S"
  format: "%m/%d/%Y %H:%M:%S"
  # Optional: Color of font
  color:
    # All Required when color is specified (default: shown below)
    red: 255
    green: 255
    blue: 255
  # Optional: Line thickness of font (default: shown below)
  thickness: 2
  # Optional: Effect of lettering (default: shown below)
  #           None (No effect),
  #           "solid" (solid background in inverse color of font)
  #           "shadow" (shadow for font)
  effect: shadow
rtmp:
  # Optional: Enable the RTMP stream (default: True)
  enabled: False
#environment_vars:
#  LIBVA_DRIVER_NAME_JELLYFIN: i965
#    LIBVA_DRIVER_NAME: i965
#    LIBVA_DRIVER_NAME: radeonsi

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:stable
    deploy:    # <------------- Add this section
      resources:
        reservations:
          devices:
            - driver: nvidia
              #device_ids: ['0'] # this is only needed when using multiple GPUs
              count: 1
              capabilities: [gpu]
    #command: ffmpeg -encoders | grep cuvid
    shm_size: "256mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modifi>
      #- /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instruc>
      #- /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your>
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /usr/share/hassio/homeassistant/frigate.yml:/config/config.yml:ro
      - /media/frigate:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"

Relevant log output

frigate  | 2023-02-12 12:43:05.761216901  192.168.86.52 - - [12/Feb/2023:12:43:05 -0700] "GET /api/stats HTTP/1.1" 200 1870 "-" "HomeAssistant/2023.2.3 aiohttp/3.8.1 Python/3.10" "-"
frigate  | 2023-02-12 12:43:07.634354217  [2023-02-12 12:43:07] watchdog.NewFront              ERROR   : Ffmpeg process crashed unexpectedly for NewFront.
frigate  | 2023-02-12 12:43:07.634374000  [2023-02-12 12:43:07] watchdog.NewFront              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2023-02-12 12:43:07.637240601  [2023-02-12 12:43:07] ffmpeg.NewFront.detect         ERROR   : [mp4 @ 0x557b45cdab00] Could not find tag for codec pcm_alaw in stream #1, codec not currently supported in container
frigate  | 2023-02-12 12:43:07.638305724  [2023-02-12 12:43:07] ffmpeg.NewFront.detect         ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
frigate  | 2023-02-12 12:43:07.640278440  [2023-02-12 12:43:07] ffmpeg.NewFront.detect         ERROR   : Error initializing output stream 0:0 -- 
frigate  | 2023-02-12 12:43:09.097340572  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.098778707  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.098789676  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.099589329  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.100304318  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.100821165  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.102015593  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.106330151  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.112088415  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.114206011  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.115515339  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.116611380  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.
frigate  | 2023-02-12 12:43:09.118827343  [2023-02-12 12:43:09] frigate.video                  ERROR   : NewFront: Unable to read frames from ffmpeg process.

FFprobe output from your camera

ffprobe version N-109851-ge506ea3ce1 Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --enable-nonfree --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --disable-static --enable-shared
  libavutil      58.  0.100 / 58.  0.100
  libavcodec     60.  1.100 / 60.  1.100
  libavformat    60.  2.100 / 60.  2.100
  libavdevice    60.  0.100 / 60.  0.100
  libavfilter     9.  1.100 /  9.  1.100
  libswscale      7.  0.100 /  7.  0.100
  libswresample   4.  9.100 /  4.  9.100

Operating system

Debian

Install method

Docker Compose

Network connection

Wired

Camera make and model

Wyze

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

yes it is supported. What are you trying to transcode? You are using the substream which is most likely already in h264 so it's really not clear what is trying to be accomplished here.

NickM-27 commented 1 year ago

also your ffprobe output only shows the header

mikerasmussen1 commented 1 year ago

this only has one camera...I have 8 connected and the CPU is at 100% so I am trying to decode and encode in the GPU so that I can offload off of the CPU.

mikerasmussen1 commented 1 year ago

I had the one camera in this file just to try to get it working...

NickM-27 commented 1 year ago

It makes no sense to encode anything unless you're trying to change it from h265 to h264. Otherwise frigate does not do any encoding

You definitely want hardware accelerated decoding but that's covered in our docs how to do that https://docs.frigate.video/configuration/hardware_acceleration

mikerasmussen1 commented 1 year ago

root@debian:~/homeassistant/frigate# ffprobe rtsp://192.168.86.52:8554/back-yard-cam ffprobe version N-109851-ge506ea3ce1 Copyright (c) 2007-2023 the FFmpeg developers built with gcc 10 (Debian 10.2.1-6) configuration: --enable-nonfree --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --disable-static --enable-shared libavutil 58. 0.100 / 58. 0.100 libavcodec 60. 1.100 / 60. 1.100 libavformat 60. 2.100 / 60. 2.100 libavdevice 60. 0.100 / 60. 0.100 libavfilter 9. 1.100 / 9. 1.100 libswscale 7. 0.100 / 7. 0.100 libswresample 4. 9.100 / 4. 9.100 [rtsp @ 0x5592c45093c0] method SETUP failed: 461 Unsupported Transport Input #0, rtsp, from 'rtsp://192.168.86.52:8554/back-yard-cam': Metadata: title : Stream Duration: N/A, start: -0.389875, bitrate: N/A Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 1920x1080, 20 fps, 20 tbr, 90k tbn Stream #0:1: Audio: pcm_alaw, 16000 Hz, mono, s16, 128 kb/s

mikerasmussen1 commented 1 year ago

Then why is ffmpeg using all of my CPU cycles?

NickM-27 commented 1 year ago

Then why is ffmpeg using all of my CPU cycles?

Well, the config you included above is not following our docs so it's difficult to say.

You also haven't defined a width / height for detect so it is being resized to 1280x720 which uses CPU as well.

I'd definitely recommend just going through the getting started guide in the docs.

mikerasmussen1 commented 1 year ago

can you point me in the right direction there? Believe me I have tried following the docs

mikerasmussen1 commented 1 year ago

what getting started docs are you referring too?

NickM-27 commented 1 year ago

https://docs.frigate.video/guides/getting_started

mikerasmussen1 commented 1 year ago

yeah I have been through those docs many times. I will try to follow it again. So are you saying that I should have the detect size match the incoming resolution to reduce CPU usage? How do I do that?

NickM-27 commented 1 year ago

yeah I have been through those docs many times. I will try to follow it again. So are you saying that I should have the detect size match the incoming resolution to reduce CPU usage? How do I do that?

Step 3 has this, which your config does not follow

Screen Shot 2023-02-12 at 14 02 42 PM

mikerasmussen1 commented 1 year ago

right, but the resolution there is 1280x720. So does this use the CPU to resize the stream to 1280x720

NickM-27 commented 1 year ago

right, but the resolution there is 1280x720. So does this use the CPU to resize the stream to 1280x720

it will resize to whatever is set there or do nothing if the size there matches your camera. Like it says in the docs, you should update that number to match your cameras resolution

mikerasmussen1 commented 1 year ago

ok I'll give that a try, thanks for your help, I'll respond here if it does not work.

mikerasmussen1 commented 1 year ago

OK, may be a different problem entirely now, but let me follow up here, happy to post new if that makes sense. Did a complete fresh install:

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:stable
    #deploy:    # <------------- Add this section
    #  resources:
    #    reservations:
    #      devices:
    #        - driver: nvidia
    #          #device_ids: ['0'] # this is only needed when using multiple GPUs
    #          count: 1
    #          capabilities: [gpu]
    #command: ffmpeg -encoders | grep cuvid
    shm_size: "256mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modifi>
      #- /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instruc>
      #- /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your>
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /usr/share/hassio/homeassistant/frigate.yml:/config/config.yml:ro
      - /media/frigate:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      #- "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: "frigatepass"
      #NVIDIA_DRIVER_CAPABILITIES: all
      #NVIDIA_VISIBLE_DEVICES: all
      LIBVA_DRIVER_NAME: i965

**Config File****

mqtt:
  host: 192.168.86.47
  port: 1883
  user: frigate
  password: frigatepass
detectors:
  coral:
    type: edgetpu
    device: usb
cameras:
    HotTubPad:
        ffmpeg:
            hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
            inputs:
              - path: rtsp://192.168.86.47:8554/hot-tub-pad # <----- Update for your camera
                roles:
                - detect
                - rtmp
        detect:
          width: 1920 # <---- update for your camera's resolution
          height: 1080 # <---- update for your camera's resolution
        record: # <----- Enable recording
          enabled: True
          retain:
            days: 30
            mode: all
          events:
            retain:
                default: 90
                mode: active_objects
        objects:
            track:
                - person
                - car
                - truck
                - cat
                - dog
    RVPad:
        ffmpeg:
            hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
            inputs:
              - path: rtsp://192.168.86.47:8554/rv-pad
                roles:
                - detect
                - rtmp
        detect:
          width: 1920 # <---- update for your camera's resolution
          height: 1080 # <---- update for your camera's resolution
        record: # <----- Enable recording
          enabled: True
          retain:
            days: 30
            mode: all
          events:
            retain:
                default: 90
                mode: active_objects
        objects:
            track:
                - person
                - car
                - truck
                - cat
                - dog
    BackDeck:
        ffmpeg:
            hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
            inputs:
              - path: rtsp://192.168.86.47:8554/back-porch
                roles:
                - detect
                - rtmp
        detect:
          width: 1920 # <---- update for your camera's resolution
          height: 1080 # <---- update for your camera's resolution
        record: # <----- Enable recording
          enabled: True
          retain:
            days: 30
            mode: all
          events:
            retain:
                default: 90
                mode: active_objects
        objects:
            track:
                - person
                - car
                - truck
                - cat
                - dog
    BackYard:
        ffmpeg:
            hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
            inputs:
              - path: rtsp://192.168.86.47:8554/back-yard-cam
                roles:
                - detect
                - rtmp
        detect:
          width: 1920 # <---- update for your camera's resolution
          height: 1080 # <---- update for your camera's resolution
        record: # <----- Enable recording
          enabled: True
          retain:
            days: 30
            mode: all
          events:
            retain:
                default: 90
                mode: active_objects
        objects:
            track:
                - person
                - car
                - truck
                - cat
                - dog
    NewFront:
        ffmpeg:
            hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
            inputs:
              - path: rtsp://admin:deadbeef11@192.168.86.94:554
                roles:
                - detect
                - rtmp
        detect:
          width: 2592 # <---- update for your camera's resolution
          height: 1944 # <---- update for your camera's resolution
        record: # <----- Enable recording
          enabled: True
          retain:
            days: 30
            mode: all
          events:
            retain:
                default: 90
                mode: active_objects
        objects:
            track:
                - person
                - car
                - truck
                - cat
                - dog
    MainFront:
        ffmpeg:
            hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
            inputs:
              - path: rtsp://admin:Sparrow_1@192.168.86.90:554
                roles:
                - detect
                - rtmp
        detect:
          width: 3840 # <---- update for your camera's resolution
          height: 2160 # <---- update for your camera's resolution
        record: # <----- Enable recording
          enabled: True
          retain:
            days: 30
            mode: all
          events:
            retain:
                default: 90
                mode: active_objects
        objects:
            track:
                - person
                - car
                - truck
                - cat
                - dog
    FrontDoor:
        ffmpeg:
            hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
            input_args:
              - -avoid_negative_ts
              - make_zero
              - -fflags
              - nobuffer
              - -flags
              - low_delay
              - -strict
              - experimental
              - -fflags
              - +genpts+discardcorrupt
              - -r
              - "15" # <---- adjust depending on your desired frame rate from the mjpeg image
              - -use_wallclock_as_timestamps
              - "1"
            inputs:
              - path: https://stream-ue1-charlie.dropcam.com:443/nexus_aac/b84d5d9f55a94a92b66afc38b1368b8f/chunklist_w516121983.m3u8?public=lx8suTOAgf
                roles:
                - detect
                - rtmp
        detect:
          width: 1152
          height: 864
          fps: 5
        record: # <----- Enable recording
          enabled: True
          retain:
            days: 30
            mode: all
          events:
            retain:
                default: 90
                mode: active_objects
        objects:
            track:
                - person
                - car
                - truck
                - cat
                - dog
rtmp:
  # Optional: Enable the RTMP stream (default: True)
  enabled: False

it’s all working, and the resolutions for "capture" are all correct but the CPU is at 100%. Is that to be expected with this many cameras? 

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND

64531 root 20 0 535704 42184 6596 R 12.5 2.3 0:16.50 ffmpeg
64660 root 20 0 539060 39144 7108 R 12.1 2.1 0:12.89 ffmpeg
64371 root 20 0 892336 33316 472 S 11.8 1.8 0:18.68 python3
64527 root 20 0 541452 44724 6316 R 11.5 2.4 0:16.19 ffmpeg
64525 root 20 0 543948 51368 11536 R 11.1 2.7 0:15.98 ffmpeg
56 root 20 0 0 0 0 S 10.5 0.0 10:07.90 kswapd0
64363 root 20 0 839876 28096 4772 S 10.2 1.5 0:18.18 python3
64798 root 20 0 331244 34948 17012 R 7.9 1.9 0:00.44 ffmpeg
64386 root 20 0 593372 83600 18468 S 6.9 4.5 0:13.10 ffmpeg
64330 root 20 0 1277480 8332 32 S 6.2 0.4 0:08.84 frigate.output
64358 root 20 0 865828 16876 280 S 6.2 0.9 0:09.72 python3
64374 root 20 0 574392 53908 5364 S 6.2 2.9 0:14.29 ffmpeg
64524 root 20 0 628232 13144 564 S 6.2 0.7 0:09.42 flask
64348 root 20 0 865828 16632 52 S 5.9 0.9 0:09.74 python3
64522 root 20 0 628232 13976 700 S 5.9 0.7 0:08.19 flask
64523 root 20 0 628296 11844 324 S 5.6 0.6 0:09.48 flask
64349 root 20 0 835452 17812 968 S 5.2 1.0 0:07.83 python3
64353 root 20 0 865828 17036 56 S 5.2 0.9 0:09.54 python3
64541 root 20 0 628232 14160 16 S 5.2 0.8 0:08.40 flask
64310 root 20 0 1640576 136160 5520 S 4.9 7.3 0:18.57 python3
64346 root 20 0 760800 19496 6888 S 3.6 1.0 0:02.21 frigate.process
64328 root 20 0 685232 17572 4340 S 3.3 0.9 0:14.33 frigate.detecto
29 root 20 0 0 0 0 S 3.0 0.0 0:07.26 kcompactd0
60880 root 20 0 679204 90812 0 S 2.3 4.9 1:19.21 python3
13755 root 20 0 721524 7164 0 S 2.0 0.4 0:59.18 rtsp-simple-ser

Top processes are 4 ffmpeg processes?

NickM-27 commented 1 year ago

It looks like you're running detection on the high res streams which is not recommended as it uses more CPU.

What cpu do you have?

mikerasmussen1 commented 1 year ago

Zimaboard,Intel Celeron N3350 Dual Core 1.1-2.4GHzMost of these cameras I don't have an option to send a lower rez stream for detect (one is a nest cam, 4 are Wyze cameras using wyze-docker-rstpSent from my iPhoneOn Feb 12, 2023, at 6:08 PM, Nicolas Mowen @.***> wrote: It looks like you're running detection on the high res streams which is not recommended as it uses more CPU. What cpu do you have?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

NickM-27 commented 1 year ago

Didn't realize wyze made a 4K camera. In any case, given your limited camera resolution options and relatively low power CPU I think this may be working as well as it can. We recommend customizable cameras because it lends itself to a much more efficient setup.

mikerasmussen1 commented 1 year ago

The Wyze cams are 1080p.What is decoding/encoding if they are all h264?Sent from my iPhoneOn Feb 12, 2023, at 6:49 PM, Nicolas Mowen @.***> wrote: Didn't realize wyze made a 4K camera. In any case, given your limited camera resolution options and relatively low power CPU I think this may be working as well as it can. We recommend customizable cameras because it lends itself to a much more efficient setup.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

NickM-27 commented 1 year ago

What camera do you have that is 4K?

And I'm not sure what you're asking, nothing is being encoded. All streams are decoded for the detect process.

mikerasmussen1 commented 1 year ago

It’s an Amcrest....So adding the Gpu would help with that thenOn Feb 12, 2023, at 7:03 PM, Nicolas Mowen @.***> wrote: What camera do you have that is 4K? And I'm not sure what you're asking, nothing is being encoded. All streams are decoded for the detect process.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

NickM-27 commented 1 year ago

If it's an amcrest then it definitely has a sub stream

NickM-27 commented 1 year ago

Also yes you're using hwaccel args for the integrated GPU not the nvidia GPU.

mikerasmussen1 commented 1 year ago

OK, so I turned off detection entirely for each camera, and I am still seeing ffmpeg using up all of the CPU:

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND

180553 root 20 0 558104 37912 6732 R 15.8 2.0 0:25.22 ffmpeg
180369 root 20 0 546488 14272 3244 R 15.5 0.8 0:30.97 ffmpeg
180546 root 20 0 561348 40624 9856 R 14.9 2.2 0:25.09 ffmpeg
180719 root 20 0 557056 39704 7612 R 14.5 2.1 0:17.12 ffmpeg
180342 root 20 0 557208 22056 7396 R 13.5 1.2 0:30.64 ffmpeg
180376 root 20 0 547156 15764 3808 R 12.2 0.8 0:26.21 ffmpeg
180287 root 20 0 1582668 50184 9588 S 6.9 2.7 0:17.64 python3
180540 root 20 0 628296 17420 1348 S 5.9 0.9 0:10.59 flask
180302 root 20 0 629256 11756 1292 S 5.6 0.6 0:10.65 flask
180539 root 20 0 629256 13332 1304 S 5.6 0.7 0:09.49 flask
180638 root 20 0 629192 15944 1228 S 5.6 0.9 0:08.41 flask
65291 root 20 0 900784 107212 5812 S 5.3 5.7 39:27.08 python3
180312 root 20 0 667388 17088 2608 S 4.3 0.9 0:10.33 frigate.detecto
178679 root 20 0 0 0 0 I 3.6 0.0 0:06.95 kworker/u8:3-events_unbound
180313 root 20 0 1238192 13196 2992 S 3.6 0.7 0:07.96 frigate.output
180325 root 20 0 736716 33352 4644 S 3.6 1.8 0:08.78 frigate.process

Did this for each camera:

HotTubPad:
    ffmpeg:
        hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
        inputs:
          - path: rtsp://192.168.86.47:8554/hot-tub-pad # <----- Update for your camera
            roles:
            - detect
            - record
    detect:
      enabled: False
    #  width: 1920 # <---- update for your camera's resolution
    #  height: 1080 # <---- update for your camera's resolution
    record: # <----- Enable recording
      enabled: True
      retain:
        days: 30
        mode: all
      events:
        retain:
            default: 90
            mode: active_objects
    objects:
        track:
            - person
            - car
            - truck
            - cat
            - dog

does that not turn off detection?

If it does, why is ffmpeg still eating up all of the resources? All of these are already in h264, which should mean no encoding needed. Is it resizing or encoding by default for some reason? Do I need to specify output arguments to ensure it does not?

NickM-27 commented 1 year ago

does that not turn off detection?

the detect process still runs as it uses motion detection to decide when to record (if recording retention is set to motion). It is also used for the live view in 0.11.

also worth noting you commented out the width / height so it will of course be resizing the stream to 1280x720 using more CPU resources

If it does, why is ffmpeg still eating up all of the resources? All of these are already in h264, which should mean no encoding needed. Is it resizing or encoding by default for some reason? Do I need to specify output arguments to ensure it does not?

you seem to be confused. You keep using the term encoding (of which there is none), BUT frigate always needs to decode the stream in order to get each frame for processing motion and object detection. which is what the detect process does.

You will be much better off using substreams from cameras (I realize some cameras don't have them, but the amcrest cameras do and that will save CPU resources https://docs.frigate.video/guides/camera_setup#choosing-a-detect-resolution

mikerasmussen1 commented 1 year ago

I have added sub streams for the amcrest cameras. It didn't seem to help that much, at least in terms of total CPU usage.

I do not have recording retention set to motion.

OK, what I am trying to determine here is can I turn off decoding entirely to ensure that no transcoding or resizing etc is happening. I have been saying "encoding" because I thought that frigate does encode if there are different codecs, or sizes from the source to the record? Is that not the case? Before I go through the trouble to add the NVIDIA to offload decoding from Frigate, I want to ensure that I can turn off detection entirely so that no decoding is happening to see what kind of improvement there should be. Is there a way to do that?

mikerasmussen1 commented 1 year ago

And BTW, thank you for your help. This is clearing up a lot of confusion. I am enjoying getting this all going, Frigate is amazing!

NickM-27 commented 1 year ago

OK, what I am trying to determine here is can I turn off decoding entirely to ensure that no transcoding or resizing etc is happening.

There is no way to do that currently. You could set the camera detect process to read from a blank video while having the recording pull from the camera, but that is just a workaround.

I have been saying "encoding" because I thought that frigate does encode if there are different codecs, or sizes from the source to the record? Is that not the case?

No, it is not the case at all. This is what frigate does for each role:

The frigate record and rtmp processes use virtually no CPU, the vast majority of the ffmpeg CPU usage comes from decoding the stream and resizIng it (if the detect -> width / height do not match the stream).

It is worth noting that frigate 0.12 (currently in beta) uses ffmpeg presets which allow for additional optimizations to use the GPU and reduce CPU resources.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.