search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
19.03k stars 1.74k forks source link

[HW Accel Support]: Nvida HW accel on bare metal HAOS install w/ Frigate addon #5384

Closed darazi closed 1 year ago

darazi commented 1 year ago

Describe the problem you are having

Thank you for this awesome product!

I have Home Assistant OS installed directly on a bare metal PC (i7 9th gen), with a PCIe nvidia gtx 780 card. I have 2 TUPs running smoothly, though I cannot get HWA to work, my camera goes green

Version

Not sure what this refers to, this is the Frigate version: 0.11.1

Frigate config file

mqtt:
  host: 192.168.1.xxx
  user: xxx
  password: xxx

detectors:
  coral1:
    type: edgetpu
    device: usb
  coral2:
    type: edgetpu
    device: usb:1

cameras:
  FrontDoor:
    ffmpeg:
      hwaccel_args: -c:v h264_cuvid
      inputs:
        - path: rtsp://xxx:xxx@10.1.1.x:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - rtmp
    detect:
      width: 3840
      height: 2160
      fps: 10
    objects:
      filters:
        person:
          threshold: 0.7
      track:
        - person
        - cat
    motion:
      threshold: 25
      mask:
        - 3840,0,3840,204,3174,193,2924,209,2785,286,2295,252,1814,245,1319,299,0,523,0,0
    zones:
      front_door_zone_0:
        coordinates: 3840,2160,3840,36,3194,489,2615,258,2197,245,1520,283,886,353,564,415,0,77,0,2160
    snapshots:
      retain:
        default: 1
      required_zones: [front_door_zone_0]

  Garage:
    ffmpeg:
#      hwaccel_args: -c:v h264_cuvid
      inputs:
        - path: rtsp://xxx:xxx@10.1.1.x:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - rtmp
    detect:
      width: 3840
      height: 2160
      fps: 10
    objects:
      filters:
        person:
          threshold: 0.7
      track:
        - person
        - cat
    motion:
      threshold: 25
      mask:
        - 3840,0,3161,551,2463,538,1747,577,971,675,0,863,0,0
    zones:
      garage_zone_0:
        coordinates: 0,2160,0,894,1984,487,2553,448,2947,435,3359,412,3840,554,3840,2160
    snapshots:
      retain:
        default: 1
      required_zones: [garage_zone_0]

  HouseFront:
    ffmpeg:
#      hwaccel_args: -c:v h264_cuvid
      inputs:
        - path: rtsp://xxx:xxx@10.1.1.x:554/cam/realmonitor?channel=1&subtype=2
          roles:
            - detect
            - rtmp
    detect:
      width: 1920
      height: 1080
      fps: 10
    objects:
      filters:
        person:
          threshold: 0.7
      track:
        - person
        - cat
    motion:
      threshold: 25
      mask:
        - 647,198,1417,231,1920,327,1920,0,0,0,0,251,393,148
    zones:
      house_front_zone_0:
        coordinates: 0,1080,0,273,95,234,183,206,537,213,854,204,1190,209,1430,232,1718,285,1920,331,1920,1080
    snapshots:
      retain:
        default: 1
      required_zones: [house_front_zone_0]

  GuestRoom:
    ffmpeg:
#      hwaccel_args: -c:v h264_cuvid
      inputs:
        - path: rtsp://xxx:xxx@10.1.1.x:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - rtmp
    detect:
      width: 3840
      height: 2160
      fps: 10
    objects:
      filters:
        person:
          threshold: 0.7
      track:
        - person
        - cat
    motion:
      threshold: 25
      mask:
        - 1920,0,1920,353,1726,374,1376,259,983,170,684,142,406,134,317,118,191,116,125,160,0,138,0,0
        - 122,229,31,291,21,219,0,173,59,169,72,196
    zones:
      guest_room_zone_0:
        coordinates: 1920,1080,1920,446,1681,358,1440,280,1141,197,772,147,515,131,325,139,169,133,0,173,0,1080
    snapshots:
      retain:
        default: 1
      required_zones: [guest_room_zone_0]

  WestFacingFront:
    ffmpeg:
#      hwaccel_args: -c:v h264_cuvid
      inputs:
        - path: rtsp://xxx:xxx@10.1.1.x:554/cam/realmonitor?channel=1&subtype=2
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 10
    objects:
      track:
        - person
        - cat
    motion:
      threshold: 25
      mask:
        - 0,0,0,356,167,365,252,298,432,152,604,21,902,21,905,60,1242,68,1240,0
    snapshots:
      retain:
        default: 1

  EastBackYard:
    ffmpeg:
#      hwaccel_args: -c:v h264_cuvid
      inputs:
        - path: rtsp://xxx:xxx@10.1.1.x:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - rtmp
    detect:
      width: 3840
      height: 2160
      fps: 10
    objects:
      track:
        - person
        - cat
    motion:
      contour_area: 45
      threshold: 25
      mask:
        - 3840,0,3840,500,3483,301,2440,93,2437,0
    snapshots:
      retain:
        default: 1

  WestBackYard:
    ffmpeg:
#      hwaccel_args: -c:v h264_cuvid
      inputs:
        - path: rtsp://xxx:xxx@10.1.1.x:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - rtmp
    detect:
      width: 3840
      height: 2160
      fps: 10
    objects:
      filters:
        person:
          threshold: 0.8
      track:
        - person
        - cat
    motion:
      contour_area: 50
      threshold: 25
      mask:
        - 2452,415,3840,546,3840,0,1394,0
    snapshots:
      retain:
        default: 1

  EastFacingFront:
    ffmpeg:
#      hwaccel_args: -c:v h264_cuvid
      inputs:
        - path: rtsp://xxx:xxx@10.1.1.x:554/cam/realmonitor?channel=1&subtype=2
          roles:
            - detect
            - rtmp
    detect:
      width: 1280
      height: 720
      fps: 10
    objects:
      track:
        - person
        - cat
    motion:
      threshold: 25
      mask:
        - 623,190,879,174,1129,367,1280,329,1280,0,486,0
    snapshots:
      retain:
        default: 1

docker-compose file or Docker CLI command

Bare Metal install of HAOS

Relevant log output

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2023-02-04 18:23:09] frigate.app                    INFO    : Starting Frigate (0.11.1-2eada21)
[2023-02-04 18:23:09] frigate.app                    INFO    : Creating directory: /tmp/cache
Starting migrations
[2023-02-04 18:23:09] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2023-02-04 18:23:09] peewee_migrate                 INFO    : There is nothing to migrate
[2023-02-04 18:23:09] detector.coral1                INFO    : Starting detection process: 224
[2023-02-04 18:23:09] detector.coral2                INFO    : Starting detection process: 226
[2023-02-04 18:23:09] frigate.app                    INFO    : Output process started: 229
[2023-02-04 18:23:09] ws4py                          INFO    : Using epoll
[2023-02-04 18:23:09] frigate.app                    INFO    : Camera processor started for FrontDoor: 237
[2023-02-04 18:23:11] frigate.video                  ERROR   : FrontDoor: Unable to read frames from ffmpeg process.
[2023-02-04 18:23:11] frigate.video                  ERROR   : FrontDoor: ffmpeg process is not running. exiting capture thread...
[2023-02-04 18:23:09] frigate.edgetpu                INFO    : Attempting to load TPU as usb:1
[2023-02-04 18:23:09] frigate.app                    INFO    : Camera processor started for Garage: 241
[2023-02-04 18:23:09] frigate.app                    INFO    : Camera processor started for HouseFront: 243
[2023-02-04 18:23:09] frigate.app                    INFO    : Camera processor started for GuestRoom: 245
[2023-02-04 18:23:09] frigate.app                    INFO    : Camera processor started for WestFacingFront: 246
[2023-02-04 18:23:09] frigate.app                    INFO    : Camera processor started for EastBackYard: 247
[2023-02-04 18:23:09] frigate.app                    INFO    : Camera processor started for WestBackYard: 249
[2023-02-04 18:23:09] frigate.app                    INFO    : Camera processor started for EastFacingFront: 250
[2023-02-04 18:23:09] frigate.app                    INFO    : Capture process started for FrontDoor: 252
[2023-02-04 18:23:09] frigate.app                    INFO    : Capture process started for Garage: 255
[2023-02-04 18:23:09] frigate.app                    INFO    : Capture process started for HouseFront: 260
[2023-02-04 18:23:09] frigate.app                    INFO    : Capture process started for GuestRoom: 265
[2023-02-04 18:23:09] frigate.app                    INFO    : Capture process started for WestFacingFront: 270
[2023-02-04 18:23:09] frigate.app                    INFO    : Capture process started for EastBackYard: 277
[2023-02-04 18:23:09] frigate.mqtt                   INFO    : Turning on snapshots for FrontDoor via mqtt
[2023-02-04 18:23:09] frigate.mqtt                   INFO    : Turning on snapshots for Garage via mqtt
[2023-02-04 18:23:09] frigate.app                    INFO    : Capture process started for WestBackYard: 282
[2023-02-04 18:23:09] frigate.mqtt                   INFO    : Turning on snapshots for HouseFront via mqtt
[2023-02-04 18:23:09] frigate.mqtt                   INFO    : Turning on snapshots for GuestRoom via mqtt
[2023-02-04 18:23:09] frigate.mqtt                   INFO    : Turning on snapshots for EastBackYard via mqtt
[2023-02-04 18:23:09] frigate.app                    INFO    : Capture process started for EastFacingFront: 287
[2023-02-04 18:23:09] frigate.mqtt                   INFO    : Turning on snapshots for WestBackYard via mqtt
[2023-02-04 18:23:09] frigate.mqtt                   INFO    : Turning on snapshots for WestFacingFront via mqtt
[2023-02-04 18:23:09] frigate.mqtt                   INFO    : Turning on snapshots for EastFacingFront via mqtt
[2023-02-04 18:23:10] ws4py                          INFO    : Using epoll
[2023-02-04 18:23:09] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2023-02-04 18:23:12] frigate.edgetpu                INFO    : TPU found
[2023-02-04 18:23:13] frigate.edgetpu                INFO    : TPU found
[2023-02-04 18:23:21] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:54546]
[2023-02-04 18:23:25] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:54546]

FFprobe output from your camera

Unable to run this command from the HA prompt.

Operating system

HassOS

Install method

HassOS Addon

Network connection

Wired

Camera make and model

Lorex E893AB

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

Nvidia hwaccel is not supported on HA OS as there is no way to instal the necessary nvidia drivers, it is one of the many reasons a native docker installation is recommended

NickM-27 commented 1 year ago

Also worth noting that running the full resolution stream for detect is discouraged as it's inefficient and generally does not produce better results.

darazi commented 1 year ago

Thanks for the quick reply @NickM-27, This setup is intended solely for facial recognition. While not very effective for face or person identification, the higher resolution images are a great help for training the deep learning engine and reducing false positives. My camera's hardware already has person and face detection and my subnets are forked into a traditional NVR for those purposes. I’m looking for GPU support in hopes of upping the resolution of my other cameras. My CPU is starting to max out.

I did initially try docker though I was not able to install a supervised version of HASS through docker, and HASS provides the best unified platform for Frigate/Doubletake/deep learning.

I might try this over again with Debian instead of Ubuntu.

NickM-27 commented 1 year ago

I mean, I run compreface, doubletake. Frigate, ha, etc in docker. I see 0 reason to need to use HA supervised to be honest. I'd much rather have the direct control offered by managing the containers myself. As I have no experience with supervised I can't be of help in answering that question.

NickM-27 commented 1 year ago

You also wouldn't be able to run Nvidia hwaccel if frigate is run as an addon anyways because there's no way to set the Nvidia runtime on the docker container

If you want to use Nvidia your only option is running frigate in native docker

darazi commented 1 year ago

Not being able to pass the Nvidia GPU through HASS to Frigate is an incredibly useful piece of information. Thank you. I'm by no means a Linux power user, hence I gravitate to the supervised version of HASS. I'll have to dig into managing my own containers.

NickM-27 commented 1 year ago

I'll go ahead and close this. Feel free to create a new issue if something else comes up