Doesn't work with Coral TPU - Githubissues

snowzach / doods2

API for detecting objects in images and video streams using Tensorflow

MIT License

232 stars 30 forks source link

Doesn't work with Coral TPU #103

Open greberg opened 6 months ago

greberg commented 6 months ago

Environment: Intel NUC7i5 USB: Coral TPU Docker container

docker-compose: doods: image: snowzach/doods2:amd64-gpu privileged: true devices:

/dev/bus/usb ports:
"8088:8088" volumes:
../doods2:/config
../doods2/models:/local environment:
CONFIG_FILE=/config/config.yaml

doods config.yaml server: host: 0.0.0.0 port: 8088 metrics: true logging: level: all doods: log: detections boxes: enabled: True boxColor: [0, 255, 0] # Or hex string "#00ff00" boxThickness: 1 fontScale: 1.2 fontColor: [0, 255, 0] # Or hex string "#00ff00" fontThickness: 1 regions: enabled: True boxColor: [255, 0, 255] # Or hex string "#ff00ff" boxThickness: 1 fontScale: 1.2 fontColor: [255, 0, 255] # Or hex string "#ff00ff" fontThickness: 1 globals: enabled: True fontScale: 1.2 fontColor: [255, 255, 0] # Or hex string "#ffff00" fontThickness: 1 detectors:

name: tflite type: tflite modelFile: local/ssd_mobilenet_v1_coco_quant_postprocess_edgetpu.tflite labelFile: local/label_file.txt hwAccel: true numThreads: 1

Startup log: 2024-03-25 16:21:06,754 - doods.doods - ERROR - Could not create detector tflite/tflite: Could not load EdgeTPU detector | 2024-03-25 16:21:06,803 - uvicorn.error - INFO - Started server process [1] | 2024-03-25 16:21:06,803 - uvicorn.error - INFO - Waiting for application startup. | 2024-03-25 16:21:06,803 - uvicorn.error - INFO - Application startup complete. | 2024-03-25 16:21:06,804 - uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8088 (Press CTRL+C to quit)

Same devices - /dev/bus/usb have worked well with Frigate before. Running lsusb gives me: Bus 001 Device 006: ID 18d1:9302 Google Inc.

snowzach commented 5 months ago

Try now

greberg commented 5 months ago

Looks like it works now. Getting this in log: INFO: Created TensorFlow Lite XNNPACK delegate for CPU. doods2_1 | 2024-04-29 07:50:25,547 - doods.doods - INFO - Registered detector type:tflite name:ssd_mobilenet_v2_edgetpu doods2_1 | 2024-04-29 07:50:25,581 - uvicorn.error - INFO - Started server process [1] doods2_1 | 2024-04-29 07:50:25,581 - uvicorn.error - INFO - Waiting for application startup. doods2_1 | 2024-04-29 07:50:25,581 - uvicorn.error - INFO - Application startup complete. doods2_1 | 2024-04-29 07:50:25,582 - uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) doods2_1 | 2024-04-29 07:50:34,036 - uvicorn.access - INFO - 10.0.1.199:64945 - "GET / HTTP/1.1" 304 doods2_1 | 2024-04-29 07:50:34,145 - uvicorn.access - INFO - 10.0.1.199:64945 - "GET /detectors HTTP/1.1" 200

What does "Created TensorFlow Lite XNNPACK delegate for CPU." mean?

Then I see something else when I try to send in a image for detection: RuntimeError: Encountered an unresolved custom op. Did you miss a custom op or delegate?Node number 4 (EdgeTpuDelegateForCustomOp) failed to invoke.

snowzach commented 5 months ago

Are you running on a 32-bit ARM device?

greberg commented 5 months ago

No, 64-bit Intel device.

snowzach commented 5 months ago

Are you using the latest image or the amd64 image?

greberg commented 5 months ago

image: snowzach/doods2:1.3.0-amd64-gpu

snowzach commented 5 months ago

Do you have an nvidia GPU? Can you try with AMD64?

greberg commented 5 months ago

No nvidia, using TPU. Absolutly, will have it a try directly, give me a sec

greberg commented 5 months ago

Unfortunately the same error: INFO: Created TensorFlow Lite XNNPACK delegate for CPU. doods2_1 | 2024-05-07 12:21:34,319 - doods.doods - INFO - Registered detector type:tflite name:ssd_mobilenet_v2_edgetpu doods2_1 | 2024-05-07 12:21:35,142 - uvicorn.error - INFO - Started server process [1] doods2_1 | 2024-05-07 12:21:35,145 - uvicorn.error - INFO - Waiting for application startup. doods2_1 | 2024-05-07 12:21:35,147 - uvicorn.error - INFO - Application startup complete. doods2_1 | 2024-05-07 12:21:35,156 - uvicorn.error - INFO - Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) doods2_1 | 2024-05-07 12:21:50,046 - uvicorn.access - INFO - 192.168.7.1:49431 - "GET / HTTP/1.1" 200 doods2_1 | 2024-05-07 12:21:50,199 - uvicorn.access - INFO - 192.168.7.1:49431 - "GET /detectors HTTP/1.1" 200 doods2_1 | 2024-05-07 12:21:50,239 - uvicorn.access - INFO - 192.168.7.1:49432 - "GET /doods.jpg HTTP/1.1" 200 doods2_1 | 2024-05-07 12:21:50,300 - uvicorn.access - INFO - 192.168.7.1:49432 - "GET /favicon.ico HTTP/1.1" 200 doods2_1 | 2024-05-07 12:22:04,917 - uvicorn.access - INFO - 192.168.7.1:49445 - "POST /image HTTP/1.1" 500 doods2_1 | 2024-05-07 12:22:04,929 - uvicorn.error - ERROR - Exception in ASGI application doods2_1 | Traceback (most recent call last): doods2_1 | File "/usr/local/lib/python3.11/dist-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi doods2_1 | result = await app(self.scope, self.receive, self.send) doods2_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doods2_1 | File "/usr/local/lib/python3.11/dist-packages/uvicorn/middleware/proxy_headers.py", line 75, in call doods2_1 | return await self.app(scope, receive, send) doods2_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doods2_1 | File "/usr/local/lib/python3.11/dist-packages/fastapi/applications.py", line 208, in call doods2_1 | await super().call(scope, receive, send) doods2_1 | File "/usr/local/lib/python3.11/dist-packages/starlette/applications.py", line 112, in call doods2_1 | await self.middleware_stack(scope, receive, send) doods2_1 | File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py", line 181, in call doods2_1 | raise exc doods2_1 | File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py", line 159, in call doods2_1 | await self.app(scope, receive, _send) doods2_1 | File "/usr/local/lib/python3.11/dist-packages/prometheus_fastapi_instrumentator/middleware.py", line 169, in call doods2_1 | raise exc doods2_1 | File "/usr/local/lib/python3.11/dist-packages/prometheus_fastapi_instrumentator/middleware.py", line 167, in call doods2_1 | await self.app(scope, receive, send_wrapper) doods2_1 | File "/usr/local/lib/python3.11/dist-packages/starlette/exceptions.py", line 82, in call doods2_1 | raise exc doods2_1 | File "/usr/local/lib/python3.11/dist-packages/starlette/exceptions.py", line 71, in call doods2_1 | await self.app(scope, receive, sender) doods2_1 | File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 656, in call doods2_1 | await route.handle(scope, receive, send) doods2_1 | File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 259, in handle doods2_1 | await self.app(scope, receive, send) doods2_1 | File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 61, in app doods2_1 | response = await func(request) doods2_1 | ^^^^^^^^^^^^^^^^^^^ doods2_1 | File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 226, in app doods2_1 | raw_response = await run_endpoint_function( doods2_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doods2_1 | File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 159, in run_endpoint_function doods2_1 | return await dependant.call(**values) doods2_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doods2_1 | File "/opt/doods/api.py", line 113, in image doods2_1 | detect_response = self.doods.detect(detect_request) doods2_1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ doods2_1 | File "/opt/doods/doods.py", line 139, in detect doods2_1 | ret = detector.detect(image) doods2_1 | ^^^^^^^^^^^^^^^^^^^^^^ doods2_1 | File "/opt/doods/detectors/tflite.py", line 92, in detect doods2_1 | interpreter.invoke() doods2_1 | File "/usr/local/lib/python3.11/dist-packages/tflite_runtime/interpreter.py", line 941, in invoke doods2_1 | self._interpreter.Invoke() doods2_1 | RuntimeError: Encountered an unresolved custom op. Did you miss a custom op or delegate?Node number 4 (EdgeTpuDelegateForCustomOp) failed to invoke.

snowzach commented 5 months ago

Not sure... try unplugging and pluggins the TPU back in?

It's working for me pulling the latest amd64. I had to restart it once.. The TPU is cranky about working right away when the drivers are first loaded.

My config file is:

doods:
  detectors:
    - name: default
      type: tflite
      modelFile: models/coco_ssd_mobilenet_v1_1.0_quant.tflite
      labelFile: models/coco_labels0.txt
    - name: edgetpu
      type: tflite
      modelFile: models/mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite
      labelFile: models/coco_labels0.txt
      hwAccel: true
      timeout: 10s
    - name: tensorflow
      type: tensorflow
      modelFile: models/faster_rcnn_inception_v2_coco_2018_01_28.pb
      labelFile: models/coco_labels1.txt