search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.76k stars 1.7k forks source link

[HW Accel Support]: Intel iGPU hw acell not working in multiple clean installs of Home Assistant + Frigate 0.12.0-beta2 addon. #5014

Closed sunnysingh-84 closed 1 year ago

sunnysingh-84 commented 1 year ago

Describe the problem you are having

Camera preview displays "No frames have been received, check error logs"

I have done some testing and setup three clean installs of Home Assistant with Frigate 0.12.0-beta2 on the following hardware and in all three instances I was able to get openvino detectors working however unable to get hardware acceleration with Intel iGPU's on all devices:

Synology DS920+: HA Supervised docker install with Frigate 0.12.0-beta2 addon using openvino detector interface speed ~150 Intel Compute Stick STK2M364CC: HAOS install with with Frigate 0.12.0-beta2 addon using openvino detector interface speed ~50 Intel NUC8I5BEK: HAOS install with Frigate 0.12.0-beta2 addon using openvino detector interface speed ~35

I tried with the following configs:

a) ffmpeg: hwaccel_args: preset-vaapi

b) environment_vars: LIBVA_DRIVER_NAME: i965 ffmpeg: hwaccel_args: preset-vaapi

c) environment_vars: LIBVA_DRIVER_NAME=i965 ffmpeg: hwaccel_args: preset-vaapi

Version

0.12.0-0DBF909

Frigate config file

mqtt:
  host: core-mosquitto
  user: user
  password: pass

ffmpeg:
  hwaccel_args: preset-intel-qsv-h264

birdseye:
  enabled: False

detectors:
  ov:
    type: openvino
    device: AUTO
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

record:
  enabled: True
  events:
    retain:
      default: 14
      mode: active_objects

snapshots:
  enabled: True

cameras:
  frontyard:
    ffmpeg:
      output_args: 
        record: preset-record-generic-audio
      inputs:
        - path: rtsp://user:pass@192.168.1.11/ISAPI/Streaming/Channels/101
          roles:
            - record
            - restream
        - path: rtsp://user:pass@192.168.1.11/ISAPI/Streaming/Channels/102
          roles:
            - detect
    detect:
      width: 1280
      height: 720
      fps: 10
    objects:
       track:
         - person
         - bicycle
         - car
         - motorcycle
  baby1_cam:
    ffmpeg:
      output_args: 
        record: preset-record-generic-audio
      inputs:
        - path: rtsp://user:password@192.168.1.10:554/Sms=9.unicast
          roles:
            - record
            - restream
            - detect
    detect:
      width: 1280
      height: 720
      fps: 10
    objects:
       track:
         - person
  baby2_cam:
    ffmpeg:
      output_args: 
        record: preset-record-generic-audio
      inputs:
        - path: rtsp://user:password@192.168.1.10:554/Sms=10.unicast
          roles:
            - record
            - restream
            - detect
    detect:
       width: 1280
       height: 720
       fps: 10
    objects:
       track:
         - person

docker-compose file or Docker CLI command

N/A

Relevant log output

2023-01-11 07:11:20.253240169  [2023-01-11 18:11:20] frigate.app                    INFO    : Starting Frigate (0.12.0-0dbf909)
2023-01-11 07:11:20.272360236  [2023-01-11 18:11:20] frigate.config                 WARNING : Customizing more than a detector model path is unsupported.
2023-01-11 07:11:20.278460505  [2023-01-11 18:11:20] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-01-11 07:11:20.278597529  [2023-01-11 18:11:20] peewee_migrate                 INFO    : Starting migrations
2023-01-11 07:11:20.282001464  [2023-01-11 18:11:20] peewee_migrate                 INFO    : There is nothing to migrate
2023-01-11 07:11:20.283712840  [2023-01-11 18:11:20] ws4py                          INFO    : Using epoll
2023-01-11 07:11:20.301082762  [2023-01-11 18:11:20] detector.ov                    INFO    : Starting detection process: 411
2023-01-11 07:11:20.305834102  [2023-01-11 18:11:20] frigate.app                    INFO    : Output process started: 413
2023-01-11 07:11:20.306026874  [2023-01-11 18:11:20] frigate.app                    INFO    : Camera processor started for frontyard: 417
2023-01-11 07:11:20.306397773  [2023-01-11 18:11:20] ws4py                          INFO    : Using epoll
2023-01-11 07:11:20.310659999  [2023-01-11 18:11:20] frigate.app                    INFO    : Camera processor started for baby1_cam: 419
2023-01-11 07:11:20.314003929  [2023-01-11 18:11:20] frigate.app                    INFO    : Camera processor started for baby2_cam: 421
2023-01-11 07:11:20.317150290  [2023-01-11 18:11:20] frigate.app                    INFO    : Capture process started for frontyard: 422
2023-01-11 07:11:20.321895620  [2023-01-11 18:11:20] frigate.app                    INFO    : Capture process started for baby1_cam: 425
2023-01-11 07:11:20.325821630  [2023-01-11 18:11:20] frigate.app                    INFO    : Capture process started for baby2_cam: 428
2023-01-11 07:11:20.555436500  [2023-01-11 18:11:20] frigate.detectors.plugins.openvino INFO    : Model Input Shape: {1, 300, 300, 3}
2023-01-11 07:11:20.555567082  [2023-01-11 18:11:20] frigate.detectors.plugins.openvino INFO    : Model Output-0 Shape: {1, 1, 100, 7}
2023-01-11 07:11:20.555649369  [2023-01-11 18:11:20] frigate.detectors.plugins.openvino INFO    : Model has 1 Output Tensors
2023-01-11 07:11:21.487893353  [2023-01-11 18:11:21] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:33972]
2023-01-11 07:11:22.331538115  [2023-01-11 18:11:22] frigate.video                  ERROR   : baby2_cam: Unable to read frames from ffmpeg process.
2023-01-11 07:11:22.331642717  [2023-01-11 18:11:22] frigate.video                  ERROR   : baby2_cam: ffmpeg process is not running. exiting capture thread...
2023-01-11 07:11:22.481975681  [2023-01-11 18:11:22] frigate.video                  ERROR   : frontyard: Unable to read frames from ffmpeg process.
2023-01-11 07:11:22.482637462  [2023-01-11 18:11:22] frigate.video                  ERROR   : frontyard: ffmpeg process is not running. exiting capture thread...
2023-01-11 07:11:22.616271653  [2023-01-11 18:11:22] frigate.video                  ERROR   : baby1_cam: Unable to read frames from ffmpeg process.
2023-01-11 07:11:22.616421403  [2023-01-11 18:11:22] frigate.video                  ERROR   : baby1_cam: ffmpeg process is not running. exiting capture thread...
2023-01-11 07:11:25.084518892  [2023-01-11 18:11:25] frigate.util                   ERROR   : Failed to initialize PMU! (Permission denied)
2023-01-11 07:11:25.084522992  
2023-01-11 07:11:40.347663712  [2023-01-11 18:11:40] watchdog.baby1_cam            ERROR   : Ffmpeg process crashed unexpectedly for baby1_cam.
2023-01-11 07:11:40.348183376  [2023-01-11 18:11:40] watchdog.baby1_cam            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-01-11 07:11:40.348828607  [2023-01-11 18:11:40] ffmpeg.baby1_cam.detect       ERROR   : Guessed Channel Layout for Input Stream #0.1 : mono
2023-01-11 07:11:40.349427347  [2023-01-11 18:11:40] ffmpeg.baby1_cam.detect       ERROR   : [AVHWDeviceContext @ 0x562e48a33f80] No VA display found for device /dev/dri/renderD128.
2023-01-11 07:11:40.349926049  [2023-01-11 18:11:40] ffmpeg.baby1_cam.detect       ERROR   : Device creation failed: -22.
2023-01-11 07:11:40.350130099  [2023-01-11 18:11:40] ffmpeg.baby1_cam.detect       ERROR   : [h264 @ 0x562e48a40040] No device available for decoder: device type vaapi needed for codec h264.
2023-01-11 07:11:40.350209495  [2023-01-11 18:11:40] ffmpeg.baby1_cam.detect       ERROR   : Device setup failed for decoder on input stream #0:0 : Invalid argument
2023-01-11 07:11:40.350297538  [2023-01-11 18:11:40] watchdog.baby2_cam            ERROR   : Ffmpeg process crashed unexpectedly for baby2_cam.
2023-01-11 07:11:40.350372346  [2023-01-11 18:11:40] watchdog.baby2_cam            ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-01-11 07:11:40.353790573  [2023-01-11 18:11:40] ffmpeg.baby2_cam.detect       ERROR   : Guessed Channel Layout for Input Stream #0.1 : mono
2023-01-11 07:11:40.353878247  [2023-01-11 18:11:40] ffmpeg.baby2_cam.detect       ERROR   : [AVHWDeviceContext @ 0x55e87d143f80] No VA display found for device /dev/dri/renderD128.
2023-01-11 07:11:40.353971461  [2023-01-11 18:11:40] ffmpeg.baby2_cam.detect       ERROR   : Device creation failed: -22.
2023-01-11 07:11:40.354075662  [2023-01-11 18:11:40] ffmpeg.baby2_cam.detect       ERROR   : [h264 @ 0x55e87d15c600] No device available for decoder: device type vaapi needed for codec

FFprobe output from your camera

Frontyard Cam:
[ { "return_code": 0, "stderr": {}, "stdout": {} }, { "return_code": 0, "stderr": {}, "stdout": {} } ]

Frontyard Cam (VLC Report):
Stream 0
Codec: H264 - MPEG-4 AVC (part 10) (h264)
Type: Video
Video resolution: 1280x720
Buffer dimensions: 1280x720
Frame rate: 25
Decoded format:
Orientation: Top left
Chroma location: Left
Stream 1
Codec: PCM ALAW (alaw)
Type: Audio
Channels: Mono
Sample rate: 8000 Hz
Bits per sample: 16

Baby Cams:
[ { "return_code": 0, "stderr": {}, "stdout": { "programs": [], "streams": [ { "avg_frame_rate": "15/1", "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10", "height": 1080, "width": 1920 }, { "avg_frame_rate": "0/0", "bit_rate": "64000", "codec_long_name": "PCM A-law / G.711 A-law" } ] } } ]

Operating system

HassOS

Install method

HassOS Addon

Network connection

Wired

Camera make and model

Hikvision & Tapo

Any other information that may be helpful

No response

sunnysingh-84 commented 1 year ago

So I managed to miss out on the part where I had to disable the protection mode on the Frigate addon page within HA.

Since doing this my CPU usage on the NUC8i5 has dropped and Inference Speed is around ~17 and on the Synology im seeing around ~40.