search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.86k stars 1.72k forks source link

[HW Accel Support]: Unable to use GTX1070 for HW Accell #6683

Closed lordratner closed 1 year ago

lordratner commented 1 year ago

Describe the problem you are having

Setup: Proxmox LXC, running frigate in Docker

I have successfully installed the Nvidia drivers and can see it with nvidia-smi in the frigate container.

When I add:

ffmpeg:
  hwaccel_args: 
    - preset-nvidia-h264

to my config, the GPU shows up on the system page, but it kills the video feeds and I get the errors in the logs.

When I hit the "VAINFO" button I get the following errors:

VAINFO OUTPUT
Return Code: 1

Process Error:

error: XDG_RUNTIME_DIR is invalid or not set in the environment.
error: can't connect to X server!
error: failed to initialize display

I get that error when I comment out the hwaccel_args and no GPU is show in the System page as well.

Version

0.12.0-DA3E197

Frigate config file

mqtt:
  host: xxxx
  port: 1883
  user: frigate
  password: xxxx

detectors:
  # coralusb:
  #   type: edgetpu
  #   device: usb
  coralpci1:
     type: edgetpu
     device: pci:0
  coralpci2:
     type: edgetpu
     device: pci:1

database:
  path: /config/frigate.db

logger:
  # Optional: Default log verbosity (default: shown below)
  default: info
  # Optional: Component specific logger overrides
#  logs:
#    frigate.event: debug

environment_vars:
  EXAMPLE_VAR: value

birdseye:
  enabled: True
  width: 1280
  height: 720
  # Optional: Encoding quality of the mpeg1 feed (default: shown below)
  # 1 is the highest quality, and 31 is the lowest. Lower quality feeds utilize less CPU resources.
  quality: 8
  # Optional: Mode of the view. Available options are: objects, motion, and continuous
  #   objects - cameras are included if they have had a tracked object within the last 30 seconds
  #   motion - cameras are included if motion was detected in the last 30 seconds
  #   continuous - all cameras are included always
  mode: motion

detect:
  fps: 5
  enabled: True
  stationary:
    interval: 50
    threshold: 50

go2rtc:
  streams:
    living_room: 
      - rtsp://xxxx:xxxx@xxxx:554//h264Preview_01_main # <- stream which supports video & aac audio. This is only supported for rtsp streams, http must use ffmpeg
      - "ffmpeg:living_room#audio=opus" # <- copy of the stream which transcodes audio to opus
    living_room_sub:
      - rtsp://frigate:xxxx@xxxx:554//h264Preview_01_sub # <- stream which supports video & aac audio. This is only supported for rtsp streams, http must use ffmpeg
    nursery:
      - rtsp://frigate:xxxx@xxxx:554//h265Preview_01_main # <- stream which supports video & aac audio. This is only supported for rtsp streams, http must use ffmpeg
      # - "ffmpeg:nursery#audio=opus" # <- copy of the stream which transcodes audio to opus
    nursery_sub:
      - rtsp://frigate:xxxx@xxxx:554//h264Preview_01_sub # <- stream which supports video & aac audio. This is only supported for rtsp streams, http must use ffmpeg

    litter_box:
      - http://xxxx/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=frigate&password=xxxx# <- stream which supports video & aac audio. This is only supported for rtsp streams, http must use ffmpeg
      - "ffmpeg:litter_box#audio=opus" # <- copy of the stream which transcodes audio to opus
    litter_box_sub:
      - http://xxxx/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=frigate&password=xxxx# <- stream which supports video & aac audio. This is only supported for rtsp streams, http must use ffmpeg

ffmpeg:
  hwaccel_args: 
    - preset-nvidia-h264

record:
  enabled: True
  expire_interval: 60
  retain:
    days: 0
    mode: all
  events:
    pre_capture: 5
    post_capture: 5
    retain:
      default: 5
      mode: all

snapshots:
  enabled: True
  timestamp: True
  bounding_box: False
  crop: True
  retain:
    default: 7

# rtmp:
#   enabled: True

live:
  height: 720
  quality: 8

timestamp_style:
  position: "bl"
  format: "%Y/%m/%d %H:%M:%S"
  color:
    red: 252
    green: 111
    blue: 3
  thickness: 3
#  effect: "shadow"

objects:
  track:
    - person
    - cat
  filters:
    cat:
      min_score: 0.5
      threshold: 0.7

cameras:
  living_room:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/living_room
          roles:
            - record
            - detect
    detect:
      width: 2560 # <---- update for your camera's resolution
      height: 1440 # <---- update for your camera's resolution
    objects:
      track:
        - person
      filters:
        person:
          mask:
            - 2287,177,2303,355,2133,328,2167,151
    zones:
      front_door:
        coordinates: 690,1068,1394,1064,1490,276,579,277
        objects:
          - person
    snapshots:
      required_zones:
        - front_door
    record:
      enabled: True
      events:
        objects:
          - person
        required_zones:
          - front_door
    mqtt:
      enabled: True
      timestamp: True
      bounding_box: False
      crop: True
      height: 500
      quality: 100
      #required_zones: []

  litter_box:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/litter_box
          roles:
            - record
            - detect
    detect:
      width: 2560 # <---- update for your camera's resolution
      height: 1920 # <---- update for your camera's resolution
    objects:
      track:
        - cat
    zones:
      entrance:
        coordinates: 1662,865,1690,674,1763,543,1741,262,1589,82,1306,0,966,116,830,427,935,677,982,892
        objects:
          - cat
      litter_room:
        coordinates: 0,0,2560,0,2560,1920,0,1920
#         coordinates: 1251,743,1234,596,1267,480,1354,384,1465,338,1591,334,1756,397,1849,460,1907,568,1931,691,1913,817,1872>
        objects:
          - cat
#          - person
    snapshots:
      required_zones:
        - entrance
    record:
      enabled: True
      events:
        objects:
          - cat
        required_zones:
          - entrance
          - litter_room

    mqtt:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True
      height: 500
      quality: 100
      #required_zones: []

  nursery:
    ffmpeg:
      input_args: preset-rtsp-restream
        # - -avoid_negative_ts
        # - make_zero
        # - -fflags
        # - +genpts+discardcorrupt
        # - -flags
        # - low_delay
        # - -strict
        # - experimental
        # - -analyzeduration
        # - 1000M
        # - -probesize
        # - 1000M
        # - -rw_timeout
        # - "5000000"
      output_args:
        # record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/nursery
          # input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      width: 2304 # <---- update for your camera's resolution
      height: 1296 # <---- update for your camera's resolution
    objects:
      track:
        - person
        - cat
    mqtt:
      enabled: True
      timestamp: True
      bounding_box: False
      crop: True
      height: 500
      quality: 100
      #required_zones: []

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true                                           
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    deploy:                                     
      resources:
        reservations:
          devices:
            - driver: nvidia
#              device_ids: ['0'] # this is only needed when using multiple GPUs
              count: 1 # number of GPUs
              capabilities: [gpu]
    shm_size: "1024mb" 
    devices:
#      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /dev/apex_0:/dev/apex_0
      - /dev/apex_1:/dev/apex_1
#      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/appdata/frigate/config:/config
      - /mnt/nvr/frigate:/media/frigate
      - type: tmpfs                                                    
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935" 
      - "8554:8554"
    environment:
      FRIGATE_RTSP_PASSWORD: "password"

Relevant log output

2023-06-02 05:08:44.163934379  [2023-06-02 05:08:44] frigate.video                  ERROR   : living_room: Unable to read frames from ffmpeg process.
2023-06-02 05:08:44.164011946  [2023-06-02 05:08:44] frigate.video                  ERROR   : living_room: ffmpeg process is not running. exiting capture thread...
2023-06-02 05:08:44.748221197  [2023-06-02 05:08:44] frigate.video                  ERROR   : nursery: Unable to read frames from ffmpeg process.
2023-06-02 05:08:44.748362975  [2023-06-02 05:08:44] frigate.video                  ERROR   : nursery: ffmpeg process is not running. exiting capture thread...
2023-06-02 05:08:45.356098071  [2023-06-02 05:08:45] frigate.video                  ERROR   : litter_box: Unable to read frames from ffmpeg process.
2023-06-02 05:08:45.356308329  [2023-06-02 05:08:45] frigate.video                  ERROR   : litter_box: ffmpeg process is not running. exiting capture thread...
2023-06-02 05:09:01.426767349  [2023-06-02 05:09:01] watchdog.nursery               ERROR   : Ffmpeg process crashed unexpectedly for nursery.
2023-06-02 05:09:01.426769433  [2023-06-02 05:09:01] watchdog.nursery               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-06-02 05:09:01.426772469  [2023-06-02 05:09:01] ffmpeg.nursery.detect          ERROR   : [NULL @ 0x560b698a7980] Unable to find a suitable output format for 'preset-nvidia-h264'
2023-06-02 05:09:01.426780043  [2023-06-02 05:09:01] ffmpeg.nursery.detect          ERROR   : preset-nvidia-h264: Invalid argument
2023-06-02 05:09:01.437946293  [2023-06-02 05:09:01] watchdog.litter_box            ERROR   : Ffmpeg process crashed unexpectedly for litter_box.
2023-06-02 05:09:01.437948697  [2023-06-02 05:09:01] watchdog.litter_box            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-06-02 05:09:01.437949569  [2023-06-02 05:09:01] ffmpeg.litter_box.detect       ERROR   : [NULL @ 0x559193852f00] Unable to find a suitable output format for 'preset-nvidia-h264'
2023-06-02 05:09:01.437950180  [2023-06-02 05:09:01] ffmpeg.litter_box.detect       ERROR   : preset-nvidia-h264: Invalid argument
2023-06-02 05:09:01.437950731  [2023-06-02 05:09:01] watchdog.living_room           ERROR   : Ffmpeg process crashed unexpectedly for living_room.
2023-06-02 05:09:01.437952134  [2023-06-02 05:09:01] watchdog.living_room           ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-06-02 05:09:01.437966872  [2023-06-02 05:09:01] ffmpeg.living_room.detect      ERROR   : [NULL @ 0x5597f0c38e00] Unable to find a suitable output format for 'preset-nvidia-h264'
2023-06-02 05:09:01.437967553  [2023-06-02 05:09:01] ffmpeg.living_room.detect      ERROR   : preset-nvidia-h264: Invalid argument

FFprobe output from your camera

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"0/0","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":1440,"width":2560},{"avg_frame_rate":"0/0","codec_long_name":"AAC(AdvancedAudioCoding)"}]}}]

Operating system

Proxmox

Install method

Docker Compose

Network connection

Wired

Camera make and model

Reolink

Any other information that may be helpful

Commenting out the hwaccel_args line makes the GPU disappear and the feeds return, but the VAINFO error doesn't change.

I think the card is setup correctly for Docker:

root@frigate:/opt# docker exec -it frigate nvidia-smi
Fri Jun  2 05:19:22 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 530.41.03              Driver Version: 530.41.03    CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                  Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf            Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce GTX 1070 Ti      Off| 00000000:13:00.0 Off |                  N/A |
|  0%   49C    P0               31W / 180W|      0MiB /  8192MiB |      2%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+

Thanks

NickM-27 commented 1 year ago

Vainfo is for Intel and amd not Nvidia so that is expected.

The reason this is failing is you are setting the hwaccel args as - preset-nvidia-h264 which is a list. You need it to be just the preset on the same line as hwaccel_args

lordratner commented 1 year ago

Thank you Nick!