search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
18.32k stars 1.67k forks source link

[EdgeTPU Support]: Unable to detect coral #5283

Closed spaceboy76 closed 1 year ago

spaceboy76 commented 1 year ago

Describe the problem you are having

Frigate doesnt see the TPU and restarts in a loop

Version

0.11.1-2eada21

Frigate config file

mqtt:
  host: 192.168.1.150
  user: mqtt
  password: passwd

# Optional: Database configuration
database:
  # The path to store the SQLite DB (default: shown below)
  path: /db/frigate.db
detectors:
  coral:
    type: edgetpu
    device: usb
# ffmpeg:
  # output_args:
    # hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
# Optional: Record configuration
# NOTE: Can be overridden at the camera level
record:
  # Optional: Enable recording (default: shown below)
  # WARNING: If recording is disabled in the config, turning it on via
  #          the UI or MQTT later will have no effect.
  # WARNING: Frigate does not currently support limiting recordings based
  #          on available disk space automatically. If using recordings,
  #          you must specify retention settings for a number of days that
  #          will fit within the available disk space of your drive or Frigate
  #          will crash.
  enabled: True
  # Optional: Number of minutes to wait between cleanup runs (default: shown below)
  # This can be used to reduce the frequency of deleting recording segments from disk if you want to minimize i/o
  expire_interval: 60
  # Optional: Retention settings for recording
  retain:
    # Optional: Number of days to retain recordings regardless of events (default: shown below)
    # NOTE: This should be set to 0 and retention should be defined in events section below
    #       if you only want to retain recordings of events.
    days: 90
    # Optional: Mode for retention. Available options are: all, motion, and active_objects
    #   all - save all recording segments regardless of activity
    #   motion - save all recordings segments with any detected motion
    #   active_objects - save all recording segments with active/moving objects
    # NOTE: this mode only applies when the days setting above is greater than 0
    mode: all
  # Optional: Event recording settings
  events:
    # Optional: Number of seconds before the event to include (default: shown below)
    pre_capture: 5
    # Optional: Number of seconds after the event to include (default: shown below)
    post_capture: 5
    # Optional: Objects to save recordings for. (default: all tracked objects)
    objects:
      - person
    # Optional: Restrict recordings to objects that entered any of the listed zones (default: no required zones)
    required_zones: []
    # Optional: Retention settings for recordings of events
    retain:
      # Required: Default retention days (default: shown below)
      default: 90
      # Optional: Mode for retention. (default: shown below)
      #   all - save all recording segments for events regardless of activity
      #   motion - save all recordings segments for events with any detected motion
      #   active_objects - save all recording segments for event with active/moving objects
      #
      # NOTE: If the retain mode for the camera is more restrictive than the mode configured
      #       here, the segments will already be gone by the time this mode is applied.
      #       For example, if the camera retain mode is "motion", the segments without motion are
      #       never stored, so setting the mode to "all" here won't bring them back.
      mode: motion
      # Optional: Per object retention days
      objects:
        person: 90
# Optional: Configuration for the jpg snapshots written to the clips directory for each event
# NOTE: Can be overridden at the camera level
snapshots:
  # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
  # This value can be set via MQTT and will be updated in startup based on retained value
  enabled: True
  # Optional: save a clean PNG copy of the snapshot image (default: shown below)
  clean_copy: True
  # Optional: print a timestamp on the snapshots (default: shown below)
  timestamp: False
  # Optional: draw bounding box on the snapshots (default: shown below)
  bounding_box: False
  # Optional: crop the snapshot (default: shown below)
  crop: False
  # Optional: height to resize the snapshot to (default: original size)
  height: 175
  # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
  required_zones: []
  # Optional: Camera override for retention settings (default: global values)
  retain:
    # Required: Default retention days (default: shown below)
    default: 90
    # Optional: Per object retention days
    objects:
      person: 90
cameras:
  front:
    ffmpeg:
      inputs:
      - path: rtsp://ubnt:c0G1$M&jwlH$@192.168.1.90:554/s0
        roles:
          - record
      - path: rtsp://ubnt:c0G1$M&jwlH$@192.168.1.90:554/s2
        roles:
          - rtmp
      - path: rtsp://ubnt:c0G1$M&jwlH$@192.168.1.90:554/s1
        roles:
          - detect
    rtmp:
      enabled: True # <-- RTMP should be disabled if your stream is not H264
    detect:
      width: 640 # <---- update for your camera's resolution
      height: 360 # <---- update for your camera's resolution
  door:
    ffmpeg:
      inputs:
      - path: rtsp://ubnt:c0G1$M&jwlH$@192.168.1.92:554/s0
        roles:
          - record
      - path: rtsp://ubnt:c0G1$M&jwlH$@192.168.1.92:554/s2
        roles:
          - rtmp
      - path: rtsp://ubnt:c0G1$M&jwlH$@192.168.1.92:554/s1
        roles:
          - detect
    rtmp:
      enabled: True # <-- RTMP should be disabled if your stream is not H264
    detect:
      width: 640 # <---- update for your camera's resolution
      height: 360 # <---- update for your camera's resolution
  rear: # <------ Name the camera
    ffmpeg:
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -tag:v hvc1 -bsf:v hevc_mp4toannexb -c:a aac
        rtmp: -c:v copy -c:a aac -f flv
      inputs:
      - path: rtsp://admin:vBN8w79muNjb@192.168.1.94:554/H264/ch1/main/av_stream # <----- Update for your camera
        roles:
         - record
      - path: rtsp://admin:vBN8w79muNjb@192.168.1.94:554/H264/ch1/sub/av_stream
        roles:
         - detect
    rtmp:
      enabled: False # <-- RTMP should be disabled if your stream is not H264
    detect:
      width: 640 # <---- update for your camera's resolution
      height: 480 # <---- update for your camera's resolution

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: blakeblackshear/frigate:stable
    shm_size: "256mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      # - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /synology-nas/frigate/config.yml:/config/config.yml:ro
      - /synology-nas/frigate:/media/frigate:rw
      - /home/mark/frigate/db:/db:rw
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: "password"
      LIBVA_DRIVER_NAME: i965

Relevant log output

2023-01-28 23:31:41 [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
2023-01-28 23:31:41 [s6-init] ensuring user provided files have correct perms...exited 0.
2023-01-28 23:31:41 [fix-attrs.d] applying ownership & permissions fixes...
2023-01-28 23:31:41 [fix-attrs.d] done.
2023-01-28 23:31:41 [cont-init.d] executing container initialization scripts...
2023-01-28 23:31:41 [cont-init.d] done.
2023-01-28 23:31:41 [services.d] starting services
2023-01-28 23:31:41 [services.d] done.
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.app                    INFO    : Starting Frigate (0.11.1-2eada21)
2023-01-28 23:31:43 Starting migrations
2023-01-28 23:31:43 [2023-01-28 23:31:43] peewee_migrate                 INFO    : Starting migrations
2023-01-28 23:31:43 There is nothing to migrate
2023-01-28 23:31:43 [2023-01-28 23:31:43] peewee_migrate                 INFO    : There is nothing to migrate
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.app                    INFO    : Output process started: 213
2023-01-28 23:31:43 [2023-01-28 23:31:43] ws4py                          INFO    : Using epoll
2023-01-28 23:31:43 [2023-01-28 23:31:43] detector.coral                 INFO    : Starting detection process: 212
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.app                    INFO    : Camera processor started for front: 218
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.edgetpu                INFO    : Attempting to load TPU as usb
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.edgetpu                ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
2023-01-28 23:31:43 Process detector:coral:
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.app                    INFO    : Camera processor started for door: 221
2023-01-28 23:31:43 Traceback (most recent call last):
2023-01-28 23:31:43   File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-01-28 23:31:43     delegate = Delegate(library, options)
2023-01-28 23:31:43   File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-01-28 23:31:43     raise ValueError(capture.message)
2023-01-28 23:31:43 ValueError
2023-01-28 23:31:43 
2023-01-28 23:31:43 During handling of the above exception, another exception occurred:
2023-01-28 23:31:43 
2023-01-28 23:31:43 Traceback (most recent call last):
2023-01-28 23:31:43   File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-01-28 23:31:43     self.run()
2023-01-28 23:31:43   File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-01-28 23:31:43     self._target(*self._args, **self._kwargs)
2023-01-28 23:31:43   File "/opt/frigate/frigate/edgetpu.py", line 135, in run_detector
2023-01-28 23:31:43     object_detector = LocalObjectDetector(
2023-01-28 23:31:43   File "/opt/frigate/frigate/edgetpu.py", line 43, in __init__
2023-01-28 23:31:43     edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-01-28 23:31:43   File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-01-28 23:31:43     raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-01-28 23:31:43 ValueError: Failed to load delegate from libedgetpu.so.1.0
2023-01-28 23:31:43 
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.app                    INFO    : Camera processor started for rear: 225
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.app                    INFO    : Capture process started for front: 227
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.app                    INFO    : Capture process started for door: 232
2023-01-28 23:31:43 [2023-01-28 23:31:43] frigate.app                    INFO    : Capture process started for rear: 236
2023-01-28 23:31:43 [2023-01-28 23:31:43] ws4py                          INFO    : Using epoll
2023-01-28 23:32:03 [2023-01-28 23:32:03] frigate.watchdog               INFO    : Detection appears to have stopped. Exiting frigate...
2023-01-28 23:32:03 [cont-finish.d] executing container finish scripts...
2023-01-28 23:32:03 [cont-finish.d] done.
2023-01-28 23:32:03 [s6-finish] waiting for services.
2023-01-28 23:32:04 [s6-finish] sending all processes the TERM signal.
2023-01-28 23:32:07 [s6-finish] sending all processes the KILL signal and exiting.

Operating system

Other Linux

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

only ever seen the coral listed under lsusb as: Bus 003 Device 006: ID 1a6e:089a Global Unichip Corp. Could it be faulty?

I'm also struggling with HW Accel although my gpu won't be up to much intel hd 6000. both of these symptoms seem to be mentioned in reference to a protection mode which needs to be turned off but I'm unsure if this is relevant if not installed on hassos.

I don't really know linux but i've read and tried most of the advice mentioned in reference to issues similar to this. cheers

NickM-27 commented 1 year ago

Bus 003 Device 006: ID 1a6e:089a Global Unichip Corp.

it will show as this for the first run then should switch to Google Inc. What does it show as on your host after trying to run frigate?

If not, then maybe your host is not able to provide enough power and you should try a different cable / different port, or use a powered USB hub.

spaceboy76 commented 1 year ago

Hi Nick, grateful for your attention.

If i understand you right i put back in the detectors section of my yml restarted the container and while it was in a start up loop ran lsusb from the terminal (not from within the container).

it still shows Bus 003 Device 006: ID 1a6e:089a Global Unichip Corp.

i have never seen it reporting as the google device i have seen referred to by other people

spaceboy76 commented 1 year ago

sorry - also should have added. host is an older intel nuc and the cable that came with the coral is plugged into a usb 3 port directly on the nuc. i've tried multiple ports on the nuc but not a powered usb hub yet. do you think that is necessary? seemed unlikely to me but i can get one if needed

NickM-27 commented 1 year ago

do you think that is necessary? seemed unlikely to me but i can get one if needed

The coral draws a lot of power, it's possible it's not getting enough power from the onboard device to fully power on.

spaceboy76 commented 1 year ago

ok no worries i can give that a try. before i do make the order on that is there any other way i can check the coral? for example if i connected it to my main desktop pc running windows would that give any indication that the coral is working or would power likely be an issue there too?

NickM-27 commented 1 year ago

You can definitely try the desktop but you'd need some way to get it to run an inference so it loads the internal driver and the name changes.

spaceboy76 commented 1 year ago

ok - ordered a usb hub. will update tomorrow

spaceboy76 commented 1 year ago

usb hub did not help, no change at all. anything else i can try or does it look like a non-functioning coral?

spaceboy76 commented 1 year ago

ordered a second coral...

S-Przybylski commented 1 year ago

Maybe I am wrong, but it could be that you have the same issue i had: I also tried using the TPU on my DS1621 Synology NAS with DSM 7.x and Frigate inside a docker container (not running in vm). This do not work at all.

Somebody found out, that it has to do with the compilation of the library (if i remember correctly). There is a modified library available.... You may search for it... I haven't tried that, I finaly moved to a Asus pn41 with a dedicated ubuntu os and frigate (not hass os).

VM on NAS was also not a path to go (the device ID changes while loading the firmware --> disconnects the device all the time)...

capandcamera commented 1 year ago

I had the same issue this morning - My set up was Proxmox -> VM -> Docker I had to install the libraries on the VM using the guide below: https://coral.ai/docs/accelerator/get-started/#runtime-on-linux

Once i did that, i restarted the VM and it worked fine.

spaceboy76 commented 1 year ago

cheers for the responses.

what about pycoral? i did notice that when i tried to run the example model after installing pycoral on a previous effort that i got errors - i believe because ubuntu has python 3.10 installed and pycoral is only compatible with 3.9 but then i read somewhere that this is all handled by the frigate container

NickM-27 commented 1 year ago

cheers for the responses.

what about pycoral? i did notice that when i tried to run the example model after installing pycoral on a previous effort that i got errors - i believe because ubuntu has python 3.10 installed and pycoral is only compatible with 3.9 but then i read somewhere that this is all handled by the frigate container

frigate container has an internal version of python 3.9, all of that is handled inside the container

spaceboy76 commented 1 year ago

but worth trying the linked library?

spaceboy76 commented 1 year ago

when i try to install the run time i get:

sudo apt-get install libedgetpu1-std Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages were automatically installed and are no longer required: libflashrom1 libftdi1-2 Use 'sudo apt autoremove' to remove them. The following NEW packages will be installed libedgetpu1-std 0 to upgrade, 1 to newly install, 0 to remove and 32 not to upgrade. 1 not fully installed or removed. Need to get 387 kB of archives. After this operation, 1,206 kB of additional disk space will be used. Get:1 https://packages.cloud.google.com/apt coral-edgetpu-stable/main amd64 libedgetpu1-std amd64 16.0 [387 kB] Fetched 387 kB in 0s (1,577 kB/s)
Selecting previously unselected package libedgetpu1-std:amd64. (Reading database ... 200639 files and directories currently installed.) Preparing to unpack .../libedgetpu1-std_16.0_amd64.deb ... Unpacking libedgetpu1-std:amd64 (16.0) ... Setting up libedgetpu1-std:amd64 (16.0) ... Setting up docker-desktop (4.16.2-95914) ... Enabling use of privileged ports by Docker Desktop kubernetes.docker.internal already in /etc/hosts ln: failed to create symbolic link '/usr/local/bin/com.docker.cli': File exists dpkg: error processing package docker-desktop (--configure): installed docker-desktop package post-installation script subprocess returned error exit status 1 Processing triggers for libc-bin (2.35-0ubuntu3.1) ... Errors were encountered while processing: docker-desktop E: Sub-process /usr/bin/dpkg returned an error code (1)

spaceboy76 commented 1 year ago

ok i resolved the error above but no change

spaceboy76 commented 1 year ago

After starting from scratch the coral is working fine, usb hub not required. This can be closed