search

blakeblackshear / frigate

NVR with realtime local object detection for IP cameras
https://frigate.video
MIT License
17.71k stars 1.62k forks source link

[Detector Support]: EdgeTPU USB Not Recognized #8955

Closed adamsih300u closed 8 months ago

adamsih300u commented 8 months ago

Describe the problem you are having

Docker container does not recognize the Edge TPU, restarts, which creates new containers in Portainer.

I have installed the Edge TPU on my Raspberry Pi 3 B+. I have tested the function of the TPU by going through the bird-recognition test they have on the Coral website. So I believe the Coral unit is working, there seems to be enough power, etc.

I have had Frigate working wonderfully on its own VM using CPU-based detection. I looked over the complexities of passing through USB to the Frigate VM, and thought it should be easier and more stable to just add a Pi in as one of my Docker Swarm nodes.

Raspberry Pi lsusb output: root@DockPi01:/home/adam/coral/pycoral# lsusb Bus 001 Device 005: ID 18d1:9302 Google Inc. Bus 001 Device 003: ID 0424:ec00 Microchip Technology, Inc. (formerly SMSC) SMSC9512/9514 Fast Ethernet Adapter Bus 001 Device 002: ID 0424:9514 Microchip Technology, Inc. (formerly SMSC) SMC9514 Hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Frigate Docker Container USB bus listing: root@ea4239ab6c81:/opt/frigate# ls /dev/bus/usb/001/ 001 002 003 005

Version

0.12.1 and 0.13.0-beta6

Frigate config file

mqtt:
  enabled: false
  host: 192.168.80.18
  port: 1883
  topic_prefix: frigate
  user: frigate
  password: frigate!

go2rtc:
  streams:
    Driveway-1:
      - rtsp://192.168.80.10:554/user=admin_password=tlJwpbo6_channel=1_stream=0.sdp?real_stream
    Garden-1:
      - rtsp://192.168.80.8:554/user=admin_password=tlJwpbo6_channel=1_stream=0&protocol=unicast&onvif=0.sdp?real_stream
    Garden-2:
      - rtsp://192.168.80.9:554/user=admin_password=tlJwpbo6_channel=1_stream=0&protocol=unicast&onvif=0.sdp?real_stream

detectors:
  coral:
    type: edgetpu
    device: usb

birdseye:
  enabled: false
  mode: motion

record:
  enabled: false
  retain:
    days: 1
    mode: motion
  events:
    retain:
      default: 90
      mode: active_objects

snapshots:
  # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
  enabled: True
  # Optional: save a clean PNG copy of the snapshot image (default: shown below)
  clean_copy: True
  # Optional: print a timestamp on the snapshots (default: shown below)
  timestamp: True
  # Optional: draw bounding box on the snapshots (default: shown below)
  bounding_box: False
  # Optional: crop the snapshot (default: shown below)
  crop: False
  # Optional: height to resize the snapshot to (default: original size)
  #height: 175
  # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
  required_zones: []
  # Optional: Camera override for retention settings (default: global values)
  retain:
    # Required: Default retention days (default: shown below)
    default: 90

timestamp_style:
  # Optional: Position of the timestamp (default: shown below)
  #           "tl" (top left), "tr" (top right), "bl" (bottom left), "br" (bottom right)
  position: "tl"
  # Optional: Format specifier conform to the Python package "datetime" (default: shown below)
  #           Additional Examples:
  #             german: "%d.%m.%Y %H:%M:%S"
  format: "%m/%d/%Y %H:%M:%S"
  # Optional: Color of font
  color:
    # All Required when color is specified (default: shown below)
    red: 255
    green: 50
    blue: 50
  # Optional: Line thickness of font (default: shown below)
  thickness: 3

objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
    - person
    - car

cameras:
  Driveway-1: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Driveway-1 # <----- The stream you want to use for detection
          roles:
            - record
            - detect

    detect:
      enabled: False # <---- disable detection until you have a working camera feed
      width: 1920 # <---- update for your camera's resolution
      height: 1080 # <---- update for your camera's resolution
    motion:
     mask:
      - 1106,0,1920,0,1920,237
      - 515,0,0,0,0,139

  Garden-1: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Garden-1 # <----- The stream you want to use for detection
          roles:
            - record
            - detect
    detect:
      enabled: False # <---- disable detection until you have a working camera feed
      width: 1920 # <---- update for your camera's resolution
      height: 1080 # <---- update for your camera's resolution
    motion:
     mask:
      - 1920,310,1616,296,1098,298,655,317,387,360,0,438,0,0,1920,0
  Garden-2: #<------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Garden-2 # <----- The stream you want to use for detection
          roles:
            - record
            - detect
    detect:
      enabled: False # <---- disable detection until you have a working camera feed
      width: 1920 # <---- update for your camera's resolution
      height: 1080 # <---- update for your camera's resolution
    motion:
      mask:
        - 0,0,1920,0,1920,393,1564,376,1120,320,725,348,0,437

docker-compose file or Docker CLI command

version: "3.9"
volumes:
  frigate:
   driver_opts:
     type: "nfs"
     o: "addr=192.168.80.104,nolock,rw,soft,nfsvers=4.2"
     device: ":/storage/docker/frigate"
  frigate-media:
   driver_opts:
     type: "nfs"
     o: "addr=192.168.80.104,nolock,rw,soft,nfsvers=4.2"
     device: ":/storage/docker/frigate-media"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:0.13.0-beta6
    shm_size: "100mb" 
    volumes:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /etc/localtime:/etc/localtime:ro
      - frigate:/config
      - frigate-media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/shm
        tmpfs:
         size: 100000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"

Relevant log output

2023-12-13 16:01:09.545045452  [2023-12-13 16:01:09] frigate.app                    INFO    : Starting Frigate (0.13.0-614a36a)
2023-12-13 16:01:09.545203095  [2023-12-13 16:01:09] frigate.app                    INFO    : Creating directory: /tmp/cache
2023-12-13 16:01:09.637879165  [2023-12-13 16:01:09] peewee_migrate.logs            INFO    : Starting migrations
2023-12-13 16:01:09.652093182  [2023-12-13 16:01:09] peewee_migrate.logs            INFO    : There is nothing to migrate
2023-12-13 16:01:09.662549421  [2023-12-13 16:01:09] frigate.app                    INFO    : Recording process started: 443
2023-12-13 16:01:09.667116991  [2023-12-13 16:01:09] frigate.app                    INFO    : go2rtc process pid: 89
2023-12-13 16:01:09.701144250  [2023-12-13 16:01:09] detector.coral                 INFO    : Starting detection process: 452
2023-12-13 16:01:09.728195481  [2023-12-13 16:01:09] frigate.app                    INFO    : Output process started: 454
2023-12-13 16:01:09.728498436  [2023-12-13 16:01:09] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2023-12-13 16:01:09.728624404  Process detector:coral:
2023-12-13 16:01:09.728797211  [2023-12-13 16:01:09] frigate.detectors.plugins.edgetpu_tfl ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
2023-12-13 16:01:09.730165892  Traceback (most recent call last):
2023-12-13 16:01:09.730234348    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-12-13 16:01:09.730237088      delegate = Delegate(library, options)
2023-12-13 16:01:09.730263599    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-12-13 16:01:09.730267172      raise ValueError(capture.message)
2023-12-13 16:01:09.730268696  ValueError
2023-12-13 16:01:09.730404115  
2023-12-13 16:01:09.730407072  During handling of the above exception, another exception occurred:
2023-12-13 16:01:09.730408329  
2023-12-13 16:01:09.730409720  Traceback (most recent call last):
2023-12-13 16:01:09.730411328    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-12-13 16:01:09.730412650      self.run()
2023-12-13 16:01:09.730414258    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-12-13 16:01:09.730417486      self._target(*self._args, **self._kwargs)
2023-12-13 16:01:09.730419277    File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
2023-12-13 16:01:09.730434574      object_detector = LocalObjectDetector(detector_config=detector_config)
2023-12-13 16:01:09.730436507    File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
2023-12-13 16:01:09.730438001      self.detect_api = create_detector(detector_config)
2023-12-13 16:01:09.730457018    File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2023-12-13 16:01:09.730458714      return api(detector_config)
2023-12-13 16:01:09.730461047    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 41, in __init__
2023-12-13 16:01:09.730462652      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-12-13 16:01:09.730479067    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-12-13 16:01:09.730480884      raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-12-13 16:01:09.730482570  ValueError: Failed to load delegate from libedgetpu.so.1.0
2023-12-13 16:01:09.730483780  
2023-12-13 16:01:09.736343158  [2023-12-13 16:01:09] frigate.app                    INFO    : Camera processor started for Driveway-1: 460
2023-12-13 16:01:09.743005563  [2023-12-13 16:01:09] frigate.app                    INFO    : Camera processor started for Garden-1: 461
2023-12-13 16:01:09.749775947  [2023-12-13 16:01:09] frigate.app                    INFO    : Camera processor started for Garden-2: 463
2023-12-13 16:01:09.757004766  [2023-12-13 16:01:09] frigate.app                    INFO    : Capture process started for Driveway-1: 465
2023-12-13 16:01:09.771182820  [2023-12-13 16:01:09] frigate.app                    INFO    : Capture process started for Garden-1: 470
2023-12-13 16:01:09.771364071  [2023-12-13 16:01:09] frigate.app                    INFO    : Capture process started for Garden-2: 474
2023-12-13 16:01:09.941827984  [2023-12-13 16:01:09] frigate.app                    WARNING : The current SHM size of 64.0MB is too small, recommend increasing it to at least 111.0MB.
2023-12-13 16:01:17.324243745  [INFO] Starting go2rtc healthcheck service...
2023-12-13 16:01:29.951536559  [2023-12-13 16:01:29] frigate.watchdog               INFO    : Detection appears to have stopped. Exiting Frigate...
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service nginx: stopping
s6-rc: info: service go2rtc-healthcheck: stopping
2023-12-13 16:01:29.975294998  [INFO] The go2rtc-healthcheck service exited with code 256 (by signal 15)
s6-rc: info: service go2rtc-healthcheck successfully stopped
2023-12-13 16:01:30.038700278  [INFO] Service NGINX exited with code 0 (by signal 0)
s6-rc: info: service nginx successfully stopped
s6-rc: info: service nginx-log: stopping
s6-rc: info: service frigate: stopping

Operating system

Debian

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

No response

NickM-27 commented 8 months ago

I have installed the Edge TPU on my Raspberry Pi 3 B+. I have tested the function of the TPU by going through the bird-recognition test they have on the Coral website. So I believe the Coral unit is working, there seems to be enough power, etc.

this does not mean there is enough power. The demos often use the lower power runtime while frigate uses the max performance runtime. As someone who uses an RPi 3b+ to test from time to time, an externally powered USB hub is required

adamsih300u commented 8 months ago

Ah, okay, I can accept that. I will procure a powered hub and give it another try. Eventually my Swarm will be Rpi 4 or 5 - something more powerful anyways, but I'm working my way toward it. Thanks for the reply!

adamsih300u commented 8 months ago

Okay; I obtained a powered USB 3.0 hub and validated the Coral works in Max mode (Max driver installed) from the Raspberry Pi OS (using the bird recognition test). The lsusb command shows the TPU hardware ID inside the Docker container. I validated inside the container that the Max driver is indeed installed. I then tried running the bird recognition test INSIDE the container, and it fails.

So, I re-thought my Docker setup - which is Docker Swarm via Portainer. Privileged execution is not normally possible on Swarm (nor is setting SHM size), so I re-configured my Pi to be a non-Swarm Docker node and deployed Frigate again. Now the Coral is recognized. So, I would say it's just general incompatibilities with Swarm; which isn't terribly necessary for Frigate anyways as Frigate needs to live on the device that has the TPU.

So, I'd call this resolved. Thanks for the reply!