[Python] Cannot select custom tpu device

[podestplatz]

Hi,

I am working with both the Coral EdgeTPU USB and PCIe accelerators. In the python detection demo the following function is responsible for creating an interpreter object.

1 def make_interpreter(model_file):
2   model_file, *device = model_file.split('@')
3   return tflite.Interpreter(
4       model_path=model_file,
5       experimental_delegates=[
6           tflite.load_delegate(EDGETPU_SHARED_LIB,
7                                {'device': device[0]} if device else {})
8       ])

Line 7 is of especial interest to me, since I would like to be able to specify the device that shall be used, instead of relying on the automatic selection that is done by libedgetpu.so.1. From reading the code, it should be possible to select a device simply by appending @<device-selector> to the path of the model file. My problem now is that I can't figure out the <device-selector>. I tried the following options:

• USB, PCIe
• 0, 1 (as defined in enum edgetpu_device_type:edgetpu_c.h
• /sys/bus/usb/devices/<corresponding usb-device>, /dev/apex_0

With every of these values I get the following error message:

Traceback (most recent call last):
  File "/home/patrick/detection/env/lib/python3.6/site-packages/tflite_runtime/interpreter.py", line 161, in load_delegate
    delegate = Delegate(library, options)
  File "/home/patrick/detection/env/lib/python3.6/site-packages/tflite_runtime/interpreter.py", line 120, in __init__
    raise ValueError(capture.message)
ValueError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "detect_image.py", line 130, in <module>
    main()
  File "detect_image.py", line 95, in main
    interpreter = make_interpreter(args.model)
  File "detect_image.py", line 62, in make_interpreter
    {'device': device[0]} if device else {})
  File "/home/patrick/detection/env/lib/python3.6/site-packages/tflite_runtime/interpreter.py", line 164, in load_delegate
    library, str(e)))
ValueError: Failed to load delegate from libedgetpu.so.1

My system runs Ubuntu 20.04 server with the PCIe accelerator built into its housing, thus simply unplugging it and attaching the USB accelerator would be no option.

Thanks in advance for your help!

Chees, podestplatz

[Namburger]

Hey! You should be able to specify devices like this:

interpreter_1 = Interpreter(model_1_path,
  experimental_delegates=[load_delegate('libedgetpu.so.1', {"device": "pci:0"})])

interpreter_2 = Interpreter(model_2_path,
  experimental_delegates=[load_delegate('libedgetpu.so.1', {"device": "usb:0"})])

FYI: We don't have an API to list all devices with the tflite API, but with edgetpu API, you can list devices like this:

from edgetpu.basic import edgetpu_utils
edge_tpus = edgetpu_utils.ListEdgeTpuPaths(
      edgetpu_utils.EDGE_TPU_STATE_UNASSIGNED)
print(edge_tpus)

Which shoulg give you something like this:

('/dev/apex_0', '/sys/bus/usb/devices/2-1')

[podestplatz]

Hey @Namburger,

thank you very much! I will check it. Just out of curiosity: in "pci:0" and "usb:0" what meaning does the 0 have? Another question: do you have a documentation on parameters the edgetpu library (libedgetpu.so.1) accepts when loaded as delegate? Because I didn't find something like that?

[Namburger]

@podestplatz sorry, should have left you the link to our docs: https://coral.ai/docs/edgetpu/multiple-edgetpu/#using-the-tensorflow-lite-python-api

Basically if you have more than one usb devices, it'll index like this usb:0 and usb:1, for me on the dev board, the edgetpu is registered as a pic device, so it's registered as pci:0

[podestplatz]

Ok I understand, quite obvious actually ^^ Thank you very much!