[Detector Support]: USB Coral Not Detected

GregHilston commented 10 months ago

Describe the problem you are having

When bringing up Frigate, I'm seeing the error output that I shared below in the Relevant log output section.

I'm not positive that this is a Frigate issue, but rather that my brand new Coral USB is either damaged, using a damaged USB port, or not playing nicely with my Proxmox VM...

Any advice would be wildly appreciated :)

Version

0.12.1-367d724

Frigate config file

mqtt:
  enabled: False

cameras:
  moto_x4: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://192.168.1.6:8080/h264_pcm.sdp # <----- The stream you want to use for detection
          roles:
            - detect
            - rtmp
            - record
    detect:
      enabled: False # <---- disable detection until you have a working camera feed
      width: 1920 # <---- update for your camera's resolution
      height: 1080 # <---- update for your camera's resolution
      fps: 15
    record:
      enabled: True
detectors:
  coral:
    type: edgetpu
    device: usb

docker-compose file or Docker CLI command

version: "3"

services:
    // ...

  frigate:
    container_name: frigate
    image: ghcr.io/blakeblackshear/frigate:stable
    privileged: true # this may not be necessary for all setups
    shm_size: "64mb" # update for your cameras based on calculation here https://docs.frigate.video/frigate/installation/#calculating-required-shm-size, especially if seeing frigate exiting wht "bus error"
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${SERVER_CONFIG_BASE}/frigate:/config
      - ${SERVER_HOME_LAB_REPO_PATH}/config/frigate/config.yml:/config/config.yml
      - ${UNRAID_WEBCAM_SHARE_MOUNT_PATH}/frigate:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: ${SECRET_FRIGATE_RTSP_PASSWORD}
    # restart: unless-stopped

    // ...

Relevant log output

2023-10-26 15:56:38.060210064  Process detector:coral:
2023-10-26 15:56:38.060238012  [2023-10-26 15:56:38] detector.coral                 INFO    : Starting detection process: 2018
2023-10-26 15:56:38.060393974  [2023-10-26 15:56:38] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2023-10-26 15:56:38.060531431  [2023-10-26 15:56:38] frigate.detectors.plugins.edgetpu_tfl ERROR   : No EdgeTPU was detected. If you do not have a Coral device yet, you must configure CPU detectors.
2023-10-26 15:56:38.061941771  Traceback (most recent call last):
2023-10-26 15:56:38.061944732    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 160, in load_delegate
2023-10-26 15:56:38.061946187      delegate = Delegate(library, options)
2023-10-26 15:56:38.061950120    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 119, in __init__
2023-10-26 15:56:38.061952187      raise ValueError(capture.message)
2023-10-26 15:56:38.061990359  ValueError
2023-10-26 15:56:38.061991803
2023-10-26 15:56:38.061993346  During handling of the above exception, another exception occurred:
2023-10-26 15:56:38.061994453
2023-10-26 15:56:38.061995659  Traceback (most recent call last):
2023-10-26 15:56:38.062005415    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-10-26 15:56:38.062006650      self.run()
2023-10-26 15:56:38.062008102    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-10-26 15:56:38.062009362      self._target(*self._args, **self._kwargs)
2023-10-26 15:56:38.062010738    File "/opt/frigate/frigate/object_detection.py", line 98, in run_detector
2023-10-26 15:56:38.062012142      object_detector = LocalObjectDetector(detector_config=detector_config)
2023-10-26 15:56:38.062018979    File "/opt/frigate/frigate/object_detection.py", line 52, in __init__
2023-10-26 15:56:38.062020366      self.detect_api = create_detector(detector_config)
2023-10-26 15:56:38.062021812    File "/opt/frigate/frigate/detectors/__init__.py", line 24, in create_detector
2023-10-26 15:56:38.062022973      return api(detector_config)
2023-10-26 15:56:38.062024392    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 37, in __init__
2023-10-26 15:56:38.062025819      edge_tpu_delegate = load_delegate("libedgetpu.so.1.0", device_config)
2023-10-26 15:56:38.062027301    File "/usr/lib/python3/dist-packages/tflite_runtime/interpreter.py", line 162, in load_delegate
2023-10-26 15:56:38.062029341      raise ValueError('Failed to load delegate from {}\n{}'.format(
2023-10-26 15:56:38.062035598  ValueError: Failed to load delegate from libedgetpu.so.1.0

Operating system

Proxmox

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

I'm running proxmox, with a debian-11-genericcloud-am64 operating system VM, which is what is housing my Frigate docker container.

I have a Coral USB passed through to the VM through the Proxmox web UI:

however, I have failed to both get Frigate to start, nor have I successfully gotten the classify_image.py with the parrot input image to work either. For example, running the following on the proxmox host or in my Debian VM, results in this output:

$ python3 classify_image.py   --model test_data/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite    --labels test_data/inat_bird_labels.txt   --input test_data/parrot.jpg
Traceback (most recent call last):
  File "/root/coral/classify_image.py", line 40, in <module>
    from pycoral.utils.edgetpu import make_interpreter
  File "/root/coral/venv/lib/python3.9/site-packages/pycoral/utils/edgetpu.py", line 24, in <module>
    from pycoral.pybind._pywrap_coral import GetRuntimeVersion as get_runtime_version
ImportError: libedgetpu.so.1: cannot open shared object file: No such file or directory

As I continue to debug this, I'm beginning to think that this is not a Frigate issue, but I figured asking here for help may be fruitful.

idontcare99999 commented 10 months ago

You'll need to passthrough the USB port or PCI device: https://community.home-assistant.io/t/google-coral-usb-frigate-proxmox/383737/36

GregHilston commented 10 months ago

Hey @idontcare99999 , thanks for the advice, but I believe I've done that? See the section above titled Any other information that may be helpful, where I described doing that and attached a screenshot. I'm also not using HA with Frigate right now.

Are you suggesting that I try to pass the USB device down via the PCI approach in Proxmox?

Thanks for your help, as I appreciate your guidance a lot

edit: this post suggests why my Debian Cloud Init VM is not showing any devices when i run $ lsusb. Additionally, /dev/bus/usb does not exist... I'm still digging, trying to figure out how to solve this...

idontcare99999 commented 10 months ago

Yes, the comments in the link I provided outlines the journey from attempting to use the USB ID, to the USB port, to the PCI ID of the controller. You cannot use the USB ID since it changes from one value after to reboot to another value after initializing the coral device. Additionally, in my experience, passing through USB devices on proxmox was noticeably slower and often errored out; I would recommend lxc which is what I am using now.

GregHilston commented 10 months ago

Okay, I follow

I think my version of Debian does not even support USBs... Hence the no output from lsusb. I'll give the pcie approach one last boy scouts try, before I throw in the towel and try the lxc approach as you mentioned.

Thanks again for your help. Ill report back and close the issue once I have some solution.

GregHilston commented 10 months ago

I ended up getting this working in a Proxmox LXC, which can now see my coral USB. i wish I could have gotten it working in my original Debian VM, but this is a great alternative :)

Thanks again for your help @idontcare99999

here's a brief ugly writeup form my personal notes. this is not complete, but may help someone in the future:

I had a pretty hard time getting the Coral TPU USB to work with Frigate. My first mistake was that my Debian VM that runs my Docker service was unable to interact with USBs. So we moved to an LXC approach, that runs Frigate in a Docker container.

There is no automatic deployment for this right now. Overall, we use Proxmox to interact with the Coral, passthrough it to the LXC for Frigate which is running in Docker. We mount a NAS share to a Proxmox mount point, pass that down to LXC and store things there.Here's the current manual process:

Mount the NAS share to Proxmox. We do this by modifying our /etc/fstab in the Proxmox host to something like this: /etc/fstab: <NAS IP>:/mnt/user/webcam /root/unraid-webcam-mountpoint nfs rw 0 0, then we run $ sudo mount -a, to mount it now.
We create an LXC for Frigate using https://tteck.github.io/Proxmox/ 's Docker - Kubernetes > Docker LXC, as this will install Docker, Docker compose, and portainer for us. At the time of writing this, you run $ bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/docker.sh)".
SSH into the LXC and copy our config.yml and docker-compose.yaml file to some new /root/frigate directory
Modify on the Proxmox host etc/pve/lxc/102.conf (where 102 is the LXC container ID), to be proxmox-pve-lxc.conf
Need to install a few packages in the LXC:

# apt install pve-headers vim tmux -y
# wget -qO - https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --no-default-keyring --keyring gnupg-ring:/etc/apt/trusted.gpg.d/google.gpg --import -
# rm /etc/apt/trusted.gpg.d/google.gpg~
# chmod 644 /etc/apt/trusted.gpg.d/google.gpg
# echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | tee /etc/apt/sources.list.d/coral-edgetpu.list
# apt-get update
# apt-get install gasket-dkms libedgetpu1-std

Run $ lsusb on Proxmox to se where the Google Coral is, and replace lxc.mount.entry: /dev/bus/usb/002/ dev/bus/usb/002/ none bind,optional,create=dir 0,0, specifically the /dev/bus/usb/002/ with the correct bus. The command lsusb, may show Bus 002 Device 007: ID 1a6e:089a Global Unichip Corp. or Bus 002 Device 010: ID 18d1:9302 Google Inc.
In the LXC, run $ docker compose up -d and verify http://<LXC IP>:5000/, or whatever the IP is, can say speak to the Moto X4 camera. Additionally, check the logs to see that it found the Coral, or does not report that it failed to find.
Set the docker-compose stack to start on boot everytime, by running this in the LXC: # systemctl enable docker
Add record in nginx for this

blakeblackshear / frigate