ValueError: Failed to load delegate from libedgetpu.so.1.0

[Eagleman7]

Describe the bug frigate fails to spot the coral device while it is forwarded to the deployment.

Version of frigate 0.8.4

Config file Include your full config file wrapped in triple back ticks.

    config: |
      database:
        path: /data/frigate.db
      mqtt:
        host: vernemq.home.svc.cluster.local
      detectors:
        coral:
          type: edgetpu
          device: usb
      cameras:
        driveway:
          ffmpeg:
            output_args:
              clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
            inputs:
              - path: rtsp://hjgdsvjhgdsjh@192.168.3.15:554/s0
                roles:
                  - detect
                  - rtmp
                  - clips
          width: 2688
          height: 1512
          fps: 5
          clips:
            enabled: True
            pre_capture: 10
            post_capture: 0
            retain:
              default: 7
          objects:
            track:
              - person
              - cat
              - dog
            filters:
              person:
                mask: 2688,0,1492,0,906,0,923,40,1048,90,1058,247,1497,255,1468,131,1582,855,1794,856,1978,1512,2688,1512
              car:
                mask: 1247,0,1093,350,1463,365,1454,245,1144,231,1231,0,2688,0,2688,1512,0,1512,0,0
              cat:
                mask: 2688,0,1492,0,906,0,923,40,1048,90,1058,247,1497,255,1468,131,1582,855,1794,856,1978,1512,2688,1512
              dog:
                mask: 2688,0,1492,0,906,0,923,40,1048,90,1058,247,1497,255,1468,131,1582,855,1794,856,1978,1512,2688,1512
          motion:
            mask: 0,0,0,1512,1974,1512,1815,883,1598,871,1474,246,1126,246,1051,244,1048,0
          snapshots:
            enabled: True
            timestamp: True
            bounding_box: True

Frigate container logs

 * Starting nginx nginx
   ...done.
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
frigate.app                    INFO    : Camera processor started for driveway: 35
frigate.app                    INFO    : Capture process started for driveway: 36
detector.coral                 INFO    : Starting detection process: 32
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.edgetpu                INFO    : No EdgeTPU detected.
Process detector:coral:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 111, in __init__
    raise ValueError(capture.message)
ValueError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/edgetpu.py", line 124, in run_detector
    object_detector = LocalObjectDetector(tf_device=tf_device, num_threads=num_threads)
  File "/opt/frigate/frigate/edgetpu.py", line 63, in __init__
    edge_tpu_delegate = load_delegate('libedgetpu.so.1.0', device_config)
  File "/usr/local/lib/python3.8/dist-packages/tflite_runtime/interpreter.py", line 154, in load_delegate
    raise ValueError('Failed to load delegate from {}\n{}'.format(
ValueError: Failed to load delegate from libedgetpu.so.1.0

lsusb on container

root@frigate-9b78547cc-tl427:/opt/frigate# lsusb
Bus 002 Device 002: ID 8087:800a
Bus 002 Device 001: ID 1d6b:0002
Bus 004 Device 002: ID 18d1:9302
Bus 004 Device 001: ID 1d6b:0003  
Bus 003 Device 005: ID 0557:2419  
Bus 003 Device 004: ID 0557:7000  
Bus 003 Device 003: ID 1cf1:0030 dresden elektronik ingenieurtechnik GmbH ConBee II
Bus 003 Device 002: ID 0658:0200  
Bus 003 Device 001: ID 1d6b:0002  
Bus 001 Device 002: ID 8087:8002  
Bus 001 Device 001: ID 1d6b:0002  

lsusb on host

[root@k8s-production-worker-2 ~]# lsusb
Bus 002 Device 002: ID 8087:800a Intel Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 002: ID 18d1:9302 Google Inc.
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 005: ID 0557:2419 ATEN International Co., Ltd
Bus 003 Device 004: ID 0557:7000 ATEN International Co., Ltd Hub
Bus 003 Device 003: ID 1cf1:0030 Dresden Elektronik
Bus 003 Device 002: ID 0658:0200 Sigma Designs, Inc. Aeotec Z-Stick Gen5 (ZW090) - UZB
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:8002 Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Computer Hardware

• OS: RHEL8
• Install method: Helm
• Virtualization: VMWare vSphere
• Coral Version: USB
• Network Setup: Wired

Camera Info:

• Manufacturer: Ubiquiti
• Model: G4 bullet
• Resolution: 2688 x 1512
• FPS: 5

Additional context Forwarded the USB devices on the motherboard to the VM, which saw it as Unicorp USB device, after initialization with this it became the Google device. It used to work, but yesterday I setup a new k8s cluster and re-did the same steps as before, the device is present in the container (18d1:9302). But frigate cannot seem to find it.

[Eagleman7]

After a reboot of the host (node) the coral USB device dissapeared, so this seems unrelated to frigate

[root@k8s-production-worker-2 ~]# lsusb
Bus 002 Device 002: ID 8087:800a Intel Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 005: ID 0557:2419 ATEN International Co., Ltd
Bus 003 Device 004: ID 0557:7000 ATEN International Co., Ltd Hub
Bus 003 Device 003: ID 1cf1:0030 Dresden Elektronik
Bus 003 Device 002: ID 0658:0200 Sigma Designs, Inc. Aeotec Z-Stick Gen5 (ZW090) - UZB
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:8002 Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

[tuomaz]

I got this error when I had missed to unplug / plug in the Coral device.

Now connect the USB Accelerator to your computer using the provided USB 3.0 cable. If you already plugged it in, remove it and replug it so the newly-installed udev rule can take effect.

From https://coral.ai/docs/accelerator/get-started/#1-install-the-edge-tpu-runtime

[Eagleman7]

Thanks @tuomaz,

I replugged my Coral USB on the server. Then it showed up again as Unicorp... I then ran the this again, which showed it as Google device. However when then forwarding the device to frigate it didnt work, after rebooting the node the entire device was gone again :(

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[doodlebro]

I am experiencing the same error on a Pi 4 with a brand new Coral TPU. Can't keep frigate running for longer than 10-15 minutes with detection on, else everything crashes and my Pi heats up, breaks SSH pipes, is basically unusable etc. Rebooting also does not work, I have to physically reset the Coral for it to come back. Edit: Sometimes even that doesn't work, and the only way to get frigate to detect the TPU again is to run the example from the Coral getting started page. Not sure what that means but it's strange behavior.

It's worth noting that I am "stress-testing" the system a bit, by constantly walking around the detection area, but this seems like our Corals may be defective, or something else is wrong with the libraries.

[blakeblackshear]

That sounds like a power related issue. Many others have to use external powered USB3 hubs to ensure enough power is supplied to the coral.

[doodlebro]

👍 running an SSD and coral is impossible (at least in the Argon One). I moved Frigate to a dedicated pi4 with no other peripherals and a 2.4A PoE hat, and things seem more stable now.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.