Darwin build fails to load delegate with USB Accelerator device

[yosyp]

Hello, and thank you for this awesome repository!

The Darwin build of libedgetpu for macOS devices as linked in the Coral documentation here does not seem to work with the Coral USB Accelerator, throwing a ValueError: Failed to load delegate from libedgetpu.1.dylib

I have tested this on 2 difference MacBook Pro laptops with USB-C ports, using macOS 10.14.5 and 10.15.7.

Full STDOUT of the error message using the example classification code from the documentation:

$ python3.7 -c 'print(__import__("tflite_runtime").__version__)'
2.1.0.post1
$ sudo python3.7 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite --labels models/inat_bird_labels.txt --input images/parrot.jpg
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 161, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 120, in __init__
    raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "classify_image.py", line 122, in <module>
    main()
  File "classify_image.py", line 99, in main
    interpreter = make_interpreter(args.model)
  File "classify_image.py", line 73, in make_interpreter
    {'device': device[0]} if device else {})
  File "/usr/local/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 164, in load_delegate
    library, str(e)))
ValueError: Failed to load delegate from libedgetpu.1.dylib

I can confirm that the Coral USB Accelerator is recognized by the MacBook using a USB-C to USB-A hub:

  Product ID:   0x089a
  Vendor ID:    0x1a6e  (Global Unichip Corp.)
  Version:  1.00
  Speed:    Up to 5 Gb/sec
  Location ID:  0x00110000 / 4
  Current Available (mA):   900
  Current Required (mA):    896
  Extra Operating Current (mA): 0

As well as using a direct USB-C to USB-C cable from the accelerator to the machine:

  Product ID:   0x089a
  Vendor ID:    0x1a6e  (Global Unichip Corp.)
  Version:  1.00
  Speed:    Up to 480 Mb/sec
  Location ID:  0x14100000 / 1
  Current Available (mA):   500
  Extra Operating Current (mA): 0

Can also confirm the locations of the libedgetpu libraries are in the local library directories with their shasums:

2b31033e73bf1bdcdecfa2d89e1ebee26aa6d9f4  /usr/local/lib/libedgetpu.1.dylib
f93854c5308751e9cbd9a40dcdf0ca7269a431c5  /usr/local/lib/libedgetpu.1.0.dylib

Building the library from source on both machines using the latest repo commit f8cac1044e3ca32b6a9c8712ac6d063e58f19fe1 results in the same behavior. I believe this may be stemming from the tflite_plugin_create_delegate() function in /edgetpu_delegate_for_custom_op_tflite_plugin.cc#L101 which calls GetEdgeTpuContext() which cannot open the recognized device. Maybe the libusb version for macOS does not expose USB devices the same way as with Linux?

[Namburger]

@yosyp I'm sure you've already looked into this, but just preliminary debugging steps; Since this message ValueError: Failed to load delegate from libedgetpu.1.dylib could appears for something as trivial as the system can't find it, can you check if /usr/local/lib is in your LD_LIBRARY_PATH? Also, does this returns anything?

python3.7 -c "from ctypes import *; cdll.LoadLibrary('libedgetpu.1.dylib'); print('success')" 

[yosyp]

@Namburger thank you for the suggestion.

$ python3.7 -c "from ctypes import *; cdll.LoadLibrary('libedgetpu.1.dylib'); print('success')"
success

It appears that $LD_LIBRARY_PATH was not set. Setting the path resolved the issue:

$ export LD_LIBRARY_PATH=/usr/local/lib/
$ python3 classify_image.py \
--model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
--labels models/inat_bird_labels.txt \
--input images/parrot.jpg
[]
models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite
----INFERENCE TIME----
Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
109.6ms
10.8ms
9.2ms
8.7ms
8.5ms
-------RESULTS--------
Ara macao (Scarlet Macaw): 0.77734

I would like to suggest adding the following line to google-coral/edgetpu/blob/master/scripts/runtime/install.sh#L88:

export LD_LIBRARY_PATH=/usr/local/lib/

and probably to also updating~/bash_profile with that variable in the script or in the documentation,

echo "export LD_LIBRARY_PATH=/usr/local/lib/" >> ~/.bash_profile

Thank you for your help Nam.

[Namburger]

@yosyp Awesome!