search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.46k stars 1.68k forks source link

[Support]: Inten NUC with USB Coral no success #6472

Closed sanderlv closed 1 year ago

sanderlv commented 1 year ago

Describe the problem you are having

See logs, it is not working.

I have succesfully passed though usb via vmware esxi:

ubuntu@frigate-02:~$ lsusb
Bus 002 Device 002: ID 18d1:9302 Google Inc.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 002: ID 0e0f:0003 VMware, Inc. Virtual Mouse
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
ubuntu@frigate-02:~$

Version

0.12.0-da3e197

Frigate config file

birdseye:
  enabled: False
#  width: 1280
#  height: 720
#  # 1 is the highest quality, and 31 is the lowest. Lower quality feeds utilize less CPU resources.
#  quality: 8
#  # Optional: Mode of the view. Available options are: objects, motion, and continuous
#  #   objects - cameras are included if they have had a tracked object within the last 30 seconds
#  #   motion - cameras are included if motion was detected in the last 30 seconds
#  #   continuous - all cameras are included always
#  mode: objects

database:
  path: /db/frigate.db

#logger:
#  default: info
#  logs:
#    frigate.record: debug

ffmpeg:
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

record:
  enabled: true
  retain:
    # Optional: Number of days to retain recordings regardless of events (default: shown below)
    # NOTE: This should be set to 0 and retention should be defined in events section below
    #       if you only want to retain recordings of events.
    days: 0
    # Optional: Mode for retention. Available options are: all, motion, and active_objects
    #   all - save all recording segments regardless of activity
    #   motion - save all recordings segments with any detected motion
    #   active_objects - save all recording segments with active/moving objects
    # NOTE: this mode only applies when the days setting above is greater than 0
    mode: all
  # Optional: Event recording settings
  events:
    retain:
      default: 14

rtmp:
  enabled: False

# MQTT Config ##########################################################################################################
mqtt:
  host: 192.168.1.142
  # Optional: port (default: shown below)
  port: 1883
  # Optional: topic prefix (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  topic_prefix: frigate-2
  # Optional: client id (default: shown below)
  # WARNING: must be unique if you are running multiple instances
  client_id: frigate-2
  # Optional: user
  user: mqtt
  # Optional: password
  # NOTE: Environment variables that begin with 'FRIGATE_' may be referenced in {}.
  #       eg. password: '{FRIGATE_MQTT_PASSWORD}'
  password: mqtt
  # Optional: interval in seconds for publishing stats (default: shown below)
  stats_interval: 60

# Cameras Config ######################################################################################################
cameras:
# FRONT Camera ########################################################################################################
  voordeur:
    ffmpeg:
     inputs:
        - path: rtsp://admin:pass@192.168.3.20:554/Streaming/Channels/101/
          roles:
            - detect
            - record
    detect:
      enabled: True
      max_disappeared: 2
      width: 2560
      height: 1440
      fps: 6
    snapshots:
      enabled: true
      crop: false
      bounding_box: True
    objects:
      track:
        - person
        - dog
        - cat
    motion:
      mask:
        - '1090,81,1216,0,1382,34,1532,99,1518,259,1426,363,1123,247'
        - '1766,498,1753,669,1778,722,1829,773,1863,825,1906,884,1924,955,1953,1065,2237,1169,2255,1128,2295,1046,2338,946,2378,841,2410,743,2215,647,2190,582,2138,512,2010,402,1859,351'

detectors:
  coral:
    type: edgetpu
    device: ""

Relevant log output

frigate  | s6-rc: info: service frigate successfully started
frigate  | s6-rc: info: service nginx: starting
frigate  | 2023-05-12 13:17:14.324349026  [INFO] Starting Frigate...
frigate  | s6-rc: info: service nginx successfully started
frigate  | 2023-05-12 13:17:14.346006691  [INFO] Starting NGINX...
frigate  | s6-rc: info: service legacy-services: starting
frigate  | s6-rc: info: service legacy-services successfully started
frigate  | 2023-05-12 13:17:14.504312493  [INFO] Starting go2rtc...
frigate  | 2023-05-12 13:17:14.605606364  13:17:14.605 INF go2rtc version 1.2.0 linux/amd64
frigate  | 2023-05-12 13:17:14.606457548  13:17:14.606 INF [api] listen addr=:1984
frigate  | 2023-05-12 13:17:14.606673840  13:17:14.606 INF [rtsp] listen addr=:8554
frigate  | 2023-05-12 13:17:14.607267512  13:17:14.607 INF [srtp] listen addr=:8443
frigate  | 2023-05-12 13:17:14.607501050  13:17:14.607 INF [webrtc] listen addr=:8555
frigate  | 2023-05-12 13:17:14.980869688  [2023-05-12 13:17:14] frigate.app                    INFO    : Starting Frigate (0.12.0-da3e197)
frigate  | 2023-05-12 13:17:14.995633171  [2023-05-12 13:17:14] peewee_migrate                 INFO    : Starting migrations
frigate  | 2023-05-12 13:17:14.998657772  [2023-05-12 13:17:14] peewee_migrate                 INFO    : There is nothing to migrate
frigate  | 2023-05-12 13:17:15.005890914  [2023-05-12 13:17:15] frigate.app                    INFO    : Output process started: 216
frigate  | 2023-05-12 13:17:15.014972742  [2023-05-12 13:17:15] frigate.app                    INFO    : Camera processor started for voordeur: 220
frigate  | 2023-05-12 13:17:15.015457045  [2023-05-12 13:17:15] frigate.app                    INFO    : Capture process started for voordeur: 221
frigate  | 2023-05-12 13:17:17.183389259  [2023-05-12 13:17:15] detector.coral                 INFO    : Starting detection process: 215
frigate  | 2023-05-12 13:17:17.183472656  [2023-05-12 13:17:15] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as
frigate  | 2023-05-12 13:17:17.186608078  [2023-05-12 13:17:17] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
frigate  | 2023-05-12 13:17:19.162852088  [2023-05-12 13:17:19] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
frigate  | 2023-05-12 13:17:19.163296774  [2023-05-12 13:17:19] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...
frigate  | 2023-05-12 13:17:24.317248000  [INFO] Starting go2rtc healthcheck service...
frigate  | 2023-05-12 13:17:35.043261855  [2023-05-12 13:17:35] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
frigate  | 2023-05-12 13:17:35.043316737  [2023-05-12 13:17:35] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2023-05-12 13:17:35.043380105  [2023-05-12 13:17:35] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x5624e7e88800] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate  | 2023-05-12 13:17:35.043405110  [2023-05-12 13:17:35] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
frigate  | 2023-05-12 13:17:35.043432653  [2023-05-12 13:17:35] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x5624e7d9d800] No device available for decoder: device type vaapi needed for codec h264.
frigate  | 2023-05-12 13:17:35.043575319  [2023-05-12 13:17:35] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate  | 2023-05-12 13:17:39.131691053  [2023-05-12 13:17:39] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
frigate  | 2023-05-12 13:17:39.132059153  [2023-05-12 13:17:39] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...
frigate  | 2023-05-12 13:17:45.047631714  [2023-05-12 13:17:45] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
frigate  | 2023-05-12 13:17:45.047679562  [2023-05-12 13:17:45] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2023-05-12 13:17:45.047778525  [2023-05-12 13:17:45] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x5624e2914700] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate  | 2023-05-12 13:17:45.047779486  [2023-05-12 13:17:45] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
frigate  | 2023-05-12 13:17:45.047782906  [2023-05-12 13:17:45] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x5624e2912300] No device available for decoder: device type vaapi needed for codec h264.
frigate  | 2023-05-12 13:17:45.047784231  [2023-05-12 13:17:45] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate  | 2023-05-12 13:17:48.833860397  [2023-05-12 13:17:48] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
frigate  | 2023-05-12 13:17:48.833913497  [2023-05-12 13:17:48] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...
frigate  | 2023-05-12 13:17:55.051731315  [2023-05-12 13:17:55] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
frigate  | 2023-05-12 13:17:55.052057445  [2023-05-12 13:17:55] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2023-05-12 13:17:55.052059308  [2023-05-12 13:17:55] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x5588d6e128c0] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate  | 2023-05-12 13:17:55.052060392  [2023-05-12 13:17:55] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
frigate  | 2023-05-12 13:17:55.052061406  [2023-05-12 13:17:55] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x5588d6e1a200] No device available for decoder: device type vaapi needed for codec h264.
frigate  | 2023-05-12 13:17:55.052062311  [2023-05-12 13:17:55] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate  | 2023-05-12 13:17:58.872576361  [2023-05-12 13:17:58] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
frigate  | 2023-05-12 13:17:58.872670831  [2023-05-12 13:17:58] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...
frigate  | 2023-05-12 13:18:05.057458834  [2023-05-12 13:18:05] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
frigate  | 2023-05-12 13:18:05.057461697  [2023-05-12 13:18:05] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2023-05-12 13:18:05.057462724  [2023-05-12 13:18:05] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x560e89d2b540] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate  | 2023-05-12 13:18:05.057463661  [2023-05-12 13:18:05] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
frigate  | 2023-05-12 13:18:05.057464652  [2023-05-12 13:18:05] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x560e89d2f200] No device available for decoder: device type vaapi needed for codec h264.
frigate  | 2023-05-12 13:18:05.057465488  [2023-05-12 13:18:05] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate  | 2023-05-12 13:18:08.860926025  [2023-05-12 13:18:08] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
frigate  | 2023-05-12 13:18:08.861119441  [2023-05-12 13:18:08] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...
frigate  | 2023-05-12 13:18:15.059773059  [2023-05-12 13:18:15] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
frigate  | 2023-05-12 13:18:15.061684249  [2023-05-12 13:18:15] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2023-05-12 13:18:15.061687451  [2023-05-12 13:18:15] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x55dfa30630c0] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate  | 2023-05-12 13:18:15.061688706  [2023-05-12 13:18:15] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
frigate  | 2023-05-12 13:18:15.061689731  [2023-05-12 13:18:15] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x55dfa2c98480] No device available for decoder: device type vaapi needed for codec h264.
frigate  | 2023-05-12 13:18:15.061690605  [2023-05-12 13:18:15] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate  | 2023-05-12 13:18:18.821523597  [2023-05-12 13:18:18] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
frigate  | 2023-05-12 13:18:18.821939876  [2023-05-12 13:18:18] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...

FFprobe output from your camera

ffprobe version 4.2.7-0ubuntu0.1 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[rtsp @ 0x55fba149cf00] max delay reached. need to consume packet
[rtsp @ 0x55fba149cf00] RTP: missed 208 packets
[h264 @ 0x55fba14a09c0] left block unavailable for requested intra mode
[h264 @ 0x55fba14a09c0] error while decoding MB 0 8, bytestream 415711
[h264 @ 0x55fba14a09c0] concealing 13169 DC, 13169 AC, 13169 MV errors in I frame
Input #0, rtsp, from 'rtsp://admin:pass@192.168.3.20:554/Streaming/Channels/101/':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.080000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 2560x1440, 25 tbr, 90k tbn, 180k tbc

Frigate stats

cpu_usages  
1   
cpu "0.0"
mem "0.0"
14  
cpu "0.0"
mem "0.0"
15  
cpu "0.0"
mem "0.0"
23  
cpu "0.0"
mem "0.0"
24  
cpu "0.0"
mem "0.0"
25  
cpu "0.0"
mem "0.0"
26  
cpu "0.0"
mem "0.0"
27  
cpu "0.0"
mem "0.0"
28  
cpu "0.0"
mem "0.0"
29  
cpu "0.0"
mem "0.0"
30  
cpu "0.0"
mem "0.0"
31  
cpu "0.0"
mem "0.0"
39  
cpu "0.0"
mem "0.0"
41  
cpu "0.0"
mem "0.0"
78  
cpu "0.0"
mem "0.0"
81  
cpu "0.0"
mem "0.0"
85  
cpu "0.0"
mem "0.0"
86  
cpu "0.0"
mem "0.4"
100 
cpu "0.0"
mem "0.1"
109 
cpu "0.0"
mem "3.5"
118 
cpu "0.0"
mem "0.3"
122 
cpu "0.0"
mem "0.2"
123 
cpu "0.0"
mem "0.1"
208 
cpu "0.0"
mem "2.0"
214 
cpu "0.0"
mem "0.3"
215 
cpu "0.0"
mem "2.5"
216 
cpu "0.0"
mem "2.3"
220 
cpu "0.0"
mem "2.2"
221 
cpu "0.0"
mem "2.2"
223 
cpu "0.0"
mem "0.4"
364 
cpu "0.0"
mem "0.0"
373 
cpu "0.0"
mem "0.1"
376 
cpu "0.0"
mem "0.1"
380 
cpu "0.0"
mem "0.1"
%Cpu(s):    
cpu "id,"
mem "0.2"
MiB 
cpu "3203.6"
mem "avail"
PID 
cpu "%CPU"
mem "%MEM"
Tasks:  
cpu "stopped,"
mem "0"
top 
cpu "average:"
mem "0.04,"
detection_fps   0
detectors   
coral   
detection_start 0
inference_speed 10
pid 215
gpu_usages  
error-gpu   
gpu -1
mem -1
service 
last_updated    1683897754
latest_version  "0.12.0"
storage 
/dev/shm    
free    268.1
mount_type  "tmpfs"
total   268.4
used    0.4
/media/frigate/clips    
free    1415.4
mount_type  "ext4"
total   11487.7
used    9466.5
/media/frigate/recordings   
free    1415.4
mount_type  "ext4"
total   11487.7
used    9466.5
/tmp/cache  
free    1000
mount_type  "tmpfs"
total   1000
used    0
temperatures    {}
uptime  319
version "0.12.0-da3e197"
voordeur    
camera_fps  0
capture_pid 221
detection_enabled   1
detection_fps   0
ffmpeg_pid  365
pid 220
process_fps 0
skipped_fps 0

Operating system

Other

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

hikvision

Any other information that may be helpful

I have this camera also on another frigate instance via proxmox that works fine.

NickM-27 commented 1 year ago

Your log errors have nothing to do with the coral. Your config also does not match up with the logs given that you are using vaapi in the logs but that is not included in your config.

sanderlv commented 1 year ago

vaapi is in config.

What does it have to do with then?

I think it has to do with the other topic and my NUC12 CPU is not "supported" atm. Correct?

NickM-27 commented 1 year ago

Your device is supported without issue, you're trying to use vaapi but your GPU hasn't been passed through to frigate, that's the issue in the logs.

sanderlv commented 1 year ago

So removing

Would solve my issue?

NickM-27 commented 1 year ago

it would solve the error in the logs, but you definitely want hwaccel enabled so most likely you'll want to fix your docker or whatever config is used to run frigate

sanderlv commented 1 year ago

Not sure what to fix then... config is above

NickM-27 commented 1 year ago

your compose config is not included in this post

sanderlv commented 1 year ago

True, sorry thought is was.

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "256mb" # update for your cameras based on calculation above
    devices:
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
#      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /frigate/config/config.yml:/config/config.yml
      - /frigate/media:/media/frigate
      - /frigate/db:/db
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"
NickM-27 commented 1 year ago

Okay, what happens if you run ls /dev/dri on your host?

sanderlv commented 1 year ago 
ubuntu@frigate-02:~$ ls /dev/dri
by-path  card0  renderD128

Think i'm hitting this: https://github.com/blakeblackshear/frigate/issues/4138

Now getting:

frigate  | 2023-05-14 18:21:15.353612002  [2023-05-14 18:21:15] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
frigate  | 2023-05-14 18:21:15.353613016  [2023-05-14 18:21:15] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x56041ae1a480] No device available for decoder: device type vaapi needed for codec h264.
frigate  | 2023-05-14 18:21:15.353613942  [2023-05-14 18:21:15] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate  | 2023-05-14 18:21:19.133177848  [2023-05-14 18:21:19] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
frigate  | 2023-05-14 18:21:19.133522376  [2023-05-14 18:21:19] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...
frigate  | 2023-05-14 18:21:25.356230032  [2023-05-14 18:21:25] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
frigate  | 2023-05-14 18:21:25.356289592  [2023-05-14 18:21:25] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2023-05-14 18:21:25.356317689  [2023-05-14 18:21:25] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x561bbba4c7c0] Failed to initialise VAAPI connection: -1 (unknown libva error).
frigate  | 2023-05-14 18:21:25.356353206  [2023-05-14 18:21:25] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
frigate  | 2023-05-14 18:21:25.356395411  [2023-05-14 18:21:25] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x561bbb67a5c0] No device available for decoder: device type vaapi needed for codec h264.
frigate  | 2023-05-14 18:21:25.356398535  [2023-05-14 18:21:25] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
frigate  | 2023-05-14 18:21:29.142009254  [2023-05-14 18:21:29] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
frigate  | 2023-05-14 18:21:29.142424858  [2023-05-14 18:21:29] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...
frigate  | 2023-05-14 18:21:35.360648497  [2023-05-14 18:21:35] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
frigate  | 2023-05-14 18:21:35.362663731  [2023-05-14 18:21:35] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2023-05-14 18:21:35.362666557  [2023-05-14 18:21:35] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x560eda556c00] Failed to initialise VAAPI connection: -1 (unknown libva error).

compose:

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "256mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
#      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /frigate/config/config.yml:/config/config.yml
      - /frigate/media:/media/frigate
      - /frigate/db:/db
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"
NickM-27 commented 1 year ago

if that is the case then it is an issue with your host driver version and you should update it

sanderlv commented 1 year ago

Followed this: https://coral.ai/docs/accelerator/get-started/#1-install-the-edge-tpu-runtime

And all is fine...

2023-05-15 06:37:00.831856030  06:37:00.831 INF [rtsp] listen addr=:8554

2023-05-15 06:37:00.832426146  06:37:00.832 INF [srtp] listen addr=:8443

2023-05-15 06:37:00.832575269  06:37:00.832 INF [webrtc] listen addr=:8555

2023-05-15 06:37:01.304711695  [2023-05-15 06:37:01] frigate.app                    INFO    : Starting Frigate (0.12.0-da3e197)

2023-05-15 06:37:01.324397201  [2023-05-15 06:37:01] peewee_migrate                 INFO    : Starting migrations

2023-05-15 06:37:01.330122519  [2023-05-15 06:37:01] peewee_migrate                 INFO    : There is nothing to migrate

2023-05-15 06:37:01.339761875  [2023-05-15 06:37:01] frigate.app                    INFO    : Output process started: 217

2023-05-15 06:37:01.342564892  [2023-05-15 06:37:01] frigate.app                    INFO    : Camera processor started for voordeur: 220

2023-05-15 06:37:01.344888643  [2023-05-15 06:37:01] frigate.app                    INFO    : Capture process started for voordeur: 221

2023-05-15 06:37:03.510589901  [2023-05-15 06:37:01] detector.coral                 INFO    : Starting detection process: 216

2023-05-15 06:37:03.510723726  [2023-05-15 06:37:01] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb

2023-05-15 06:37:03.516354627  [2023-05-15 06:37:03] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found

2023-05-15 06:37:05.337008675  [2023-05-15 06:37:05] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.

2023-05-15 06:37:05.337204593  [2023-05-15 06:37:05] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...

2023-05-15 06:37:10.479257289  [INFO] Starting go2rtc healthcheck service...

2023-05-15 06:37:21.362421443  [2023-05-15 06:37:21] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.

2023-05-15 06:37:21.362514290  [2023-05-15 06:37:21] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.

2023-05-15 06:37:21.362517908  [2023-05-15 06:37:21] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x5571edcf28c0] Failed to initialise VAAPI connection: -1 (unknown libva error).

2023-05-15 06:37:21.362540490  [2023-05-15 06:37:21] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.

2023-05-15 06:37:21.362569943  [2023-05-15 06:37:21] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x5571edcfab40] No device available for decoder: device type vaapi needed for codec h264.

2023-05-15 06:37:21.362592930  [2023-05-15 06:37:21] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error

2023-05-15 06:37:25.644677305  [2023-05-15 06:37:25] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.

2023-05-15 06:37:25.645040513  [2023-05-15 06:37:25] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.

2023-05-15 06:37:25.645163750  [2023-05-15 06:37:25] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...

2023-05-15 06:37:31.366346215  [2023-05-15 06:37:31] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.

2023-05-15 06:37:31.366446676  [2023-05-15 06:37:31] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.

2023-05-15 06:37:31.366448097  [2023-05-15 06:37:31] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x55d2452e3e40] Failed to initialise VAAPI connection: -1 (unknown libva error).

2023-05-15 06:37:31.366463303  [2023-05-15 06:37:31] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.

2023-05-15 06:37:31.366492985  [2023-05-15 06:37:31] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x55d2451d3200] No device available for decoder: device type vaapi needed for codec h264.

2023-05-15 06:37:31.366515978  [2023-05-15 06:37:31] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error

2023-05-15 06:37:35.145701360  [2023-05-15 06:37:35] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.

2023-05-15 06:37:35.146091614  [2023-05-15 06:37:35] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.

2023-05-15 06:37:35.146129615  [2023-05-15 06:37:35] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...

2023-05-15 06:37:41.370470976  [2023-05-15 06:37:41] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.

2023-05-15 06:37:41.370573437  [2023-05-15 06:37:41] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.

2023-05-15 06:37:41.370574791  [2023-05-15 06:37:41] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x55ead0aeca40] Failed to initialise VAAPI connection: -1 (unknown libva error).

2023-05-15 06:37:41.370579227  [2023-05-15 06:37:41] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.

2023-05-15 06:37:41.370602853  [2023-05-15 06:37:41] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x55ead0aee200] No device available for decoder: device type vaapi needed for codec h264.
NickM-27 commented 1 year ago

Looks like the driver on the host for the GPU hasn't been updated yet

sanderlv commented 1 year ago

Again, I think I did... what else should be done than?

NickM-27 commented 1 year ago

Can you run vainfo on the host and in frigate, paste both here please

sanderlv commented 1 year ago

Like this?

Host:

ubuntu@frigate-02:~$ vainfo
Command 'vainfo' not found, but can be installed with:
sudo apt install vainfo

ubuntu@frigate-02:~$

ubuntu@frigate-02:~$ vainfo
error: can't connect to X server!
error: failed to initialize display
ubuntu@frigate-02:~$

frigate:

2023-05-15 12:20:09.823170393  [2023-05-15 12:20:09] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
2023-05-15 12:20:09.823173320  [2023-05-15 12:20:09] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-05-15 12:20:09.823174349  [2023-05-15 12:20:09] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x56354f8c9d80] Failed to initialise VAAPI connection: -1 (unknown libva error).
2023-05-15 12:20:09.823175331  [2023-05-15 12:20:09] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
2023-05-15 12:20:09.823176244  [2023-05-15 12:20:09] ffmpeg.voordeur.detect         ERROR   : [h264 @ 0x56354f8c7f00] No device available for decoder: device type vaapi needed for codec h264.
2023-05-15 12:20:09.823177005  [2023-05-15 12:20:09] ffmpeg.voordeur.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error

vainfo

2023-05-15 12:20:13.628202271  [2023-05-15 12:20:13] frigate.video                  ERROR   : voordeur: Unable to read frames from ffmpeg process.
2023-05-15 12:20:13.628789110  [2023-05-15 12:20:13] frigate.video                  ERROR   : voordeur: ffmpeg process is not running. exiting capture thread...
2023-05-15 12:20:19.827466221  [2023-05-15 12:20:19] watchdog.voordeur              ERROR   : Ffmpeg process crashed unexpectedly for voordeur.
2023-05-15 12:20:19.829360054  [2023-05-15 12:20:19] watchdog.voordeur              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-05-15 12:20:19.829362659  [2023-05-15 12:20:19] ffmpeg.voordeur.detect         ERROR   : [AVHWDeviceContext @ 0x55623de484c0] Failed to initialise VAAPI connection: -1 (unknown libva error).
2023-05-15 12:20:19.829363752  [2023-05-15 12:20:19] ffmpeg.voordeur.detect         ERROR   : Device creation failed: -5.
2023-05-15 12:20:19.829364738  [2023-05-15 12:20:19]
NickM-27 commented 1 year ago

You'll need to install it on your host then, frigate webUI has a vainfo button you can use for that

sanderlv commented 1 year ago

Frigate

{"return_code":3,"stderr":"error:XDG_RUNTIME_DIRisinvalidornotsetintheenvironment.nerror:can'tconnecttoXserver!nlibvainfo:VA-APIversion1.17.0nlibvainfo:Tryingtoopen/usr/lib/x86_64-linux-gnu/dri/vmwgfx_drv_video.sonlibvainfo:va_openDriver()returns-1nvaInitializefailedwitherrorcode-1(unknownlibvaerror),exit","stdout":""}

Host

ubuntu@frigate-02:~$ vainfo
error: can't connect to X server!
error: failed to initialize display
ubuntu@frigate-02:~$
NickM-27 commented 1 year ago

Doesn't look like any GPU drivers are installed

sanderlv commented 1 year ago

How would I do that? I never did that in proxmox....

I passed through the usb coral and selected these settings now for video: image

NickM-27 commented 1 year ago

I'm not 100% sure, you'll likely need to ask elsewhere or wait for someone who uses proxmox to explain. But the lack of vainfo makes it clear that the driver is the problem

sanderlv commented 1 year ago

I meant to say, in proxmox all worked "out of the box". Now in ESXi not yet.

In proxmox I never had to install any "drivers", not on the host as well on the guest. Just run Frigate in Docker (with complexity at that level) works fine. (Even in LXC - Thus Proxmox, LXC, Docker).

Where would a "driver" need to be installed when using esxi and ubuntu. I followed the coral installation manual and installed it in the ubuntu guast on esxi with which the coral usb is passed through.

NickM-27 commented 1 year ago

The GPU would need to be passed through as a PCIe device to the Ubuntu VM and the driver needs to be installed in Ubuntu VM.

sanderlv commented 1 year ago

The thing is, it's an Intel NUC12. There's no GPU in there. And that same is also the case with another NUC(8) which has no gpu.

Both have a usb coral attached. The proxmox one (nuc8) works fine (as described above). But not the esxi one.

What do you mean with a gpu here? Cpu? Or coral?

NickM-27 commented 1 year ago

You are incorrect, the nuc supports display output so of course it has a GPU which is integrated in the CPU.

sanderlv commented 1 year ago

Got you now, will dig into that.

So my problem with esxi is that the "gpu" (in the cpu) is not passed through to the guest.

NickM-27 commented 1 year ago

If it is not then that is definitely the problem

lamw commented 1 year ago

So my problem with esxi is that the "gpu" (in the cpu) is not passed through to the guest.

I don't know the Frigate software ... but if it requires a GPU, then it sounds like you're missing a requirement to properly run and this has nothing to do w/Coral device?

The Intel NUC can passthrough the iGPU to VM, see https://williamlam.com/2022/11/updated-findings-for-passthrough-of-intel-nuc-integrated-graphics-igpu.html

It'll add a new render and you can successfully access it as shown below (BTW this is using Intel NUC w/Ubuntu 20.04)

# vainfo --display drm --device /dev/dri/renderD129
libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.14 (libva 2.14.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 22.4.3 (74f40ee)
vainfo: Supported profile and entrypoints
      VAProfileNone                   : VAEntrypointVideoProc
      VAProfileNone                   : VAEntrypointStats
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Simple            : VAEntrypointEncSlice
....

I assume you'll need to then pass this render + Coral device into Frigate Docker/Docker Compose configuration .... but the Coral TPU is detected afaict:

frigate  | 2023-05-16 22:47:52.650948875  [2023-05-16 22:47:50] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
frigate  | 2023-05-16 22:47:52.650986331  [2023-05-16 22:47:52] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.