Request: Google Corral TPU support

[XtremeOwnageDotCom]

Add functionality similar to frigate.io, which is able to leverage a Corral TPU to do real time object detection with basic no resource usage.

In my testing on frigate, it is able to achieve detection times measured in milliseconds when combined with the Corral tpu.

With this capability for this project, combined with the robust features set of blue iris, would make for an extremely formidable nvr solution.

In current state using yolo v5 on cpu, detections can take a while and occasionally lag behind.

[erikpendragon]

yes please.

[ministryofsillywalks]

It's being worked on an should be available soon There is more discussion going on here: https://www.codeproject.com/Articles/5322557/CodeProject-AI-Server-AI-the-easy-way?msg=5935241#xx5935241xx

[Shawson]

It looks like its available on the RPi image- I'm running the linux docker image codeproject/ai-server:latest (pulled yesterday) on a normal ubuntu x64 system and, with priviledged mode enabled and mapping through /dev/bus/usb, i do see the option to add tflite object detection- the install seems to hang for a while and eventually fail, but appearing to have installed?

13:03:41:ObjectDetectionTFLite: Choosing packages from requirements.linux.txt
13:13:05:Module ObjectDetectionTFLite installed successfully.
13:13:06:ObjectDetectionTFLite: Installing Packages into Virtual Environment...
13:13:06:Module ObjectDetectionTFLite not configured to AutoStart.
13:13:06:Timed out attempting to install Module 'ObjectDetectionTFLite' ($A task was canceled.)
13:13:06:Timed out attempting to install Module 'ObjectDetectionTFLite' ($A task was canceled.)

If i do then go on to try and start it, it fails

13:16:46:Module 'ObjectDetection (TF-Lite)' (ID: ObjectDetectionTFLite)
13:16:46:Module Path:   /app/modules/ObjectDetectionTFLite
13:16:46:AutoStart:     True
13:16:46:Queue:         objectdetection_queue
13:16:46:Platforms:     windows,linux,linux-arm64,macos,macos-arm64
13:16:46:GPU:           Support enabled
13:16:46:Starting /app...nTFLite/bin/linux/python39/venv/bin/python3 "/app...nTFLite/objectdetection_tflite_adapter.py"
13:16:46:Attempting to start ObjectDetectionTFLite with /app/modules/ObjectDetectionTFLite/bin/linux/python39/venv/bin/python3 "/app/modules/ObjectDetectionTFLite/objectdetection_tflite_adapter.py"
13:16:46:Parallelism:   0
13:16:46:Accelerator:
13:16:46:Half Precis.:  enable
13:16:46:Runtime:       python39
13:16:46:Runtime Loc:   Local
13:16:46:FilePath:      objectdetection_tflite_adapter.py
13:16:46:Pre installed: False
13:16:46:Start pause:   1 sec
13:16:46:LogVerbosity:
13:16:46:Valid:         True
13:16:46:Environment Variables
13:16:46:MODELS_DIR = %CURRENT_MODULE_PATH%/assets
13:16:46:MODEL_SIZE = Medium
13:16:46:
13:16:46:Started ObjectDetection (TF-Lite) module
13:16:47:objectdetection_tflite_adapter.py: Traceback (most recent call last):
13:16:47:objectdetection_tflite_adapter.py:   File "/app/modules/ObjectDetectionTFLite/objectdetection_tflite_adapter.py", line 9, in 
13:16:47:objectdetection_tflite_adapter.py:     from request_data import RequestData
13:16:47:objectdetection_tflite_adapter.py:   File "/app/modules/ObjectDetectionTFLite/../../SDK/Python/request_data.py", line 8, in 
13:16:47:objectdetection_tflite_adapter.py:     from PIL import Image
13:16:47:objectdetection_tflite_adapter.py: ModuleNotFoundError: No module named 'PIL'
13:16:47:Module ObjectDetectionTFLite has shutdown
13:16:47:objectdetection_tflite_adapter.py: has exited

I'm assuming this is just me jumping the gun trying to enable a feature thats not ready for x64 platform? Kind of feels like its not a million miles away though..

update (just caught up on the discussion thread on the site! I'll leave this here incase the output is useful?)

[s-bed]

It appears to be possible to make it work.

docker exec -it \<container name> bash ln -s /usr/include/locale.h /usr/include/xlocale.h cd /app/modules/ObjectDetectionTFLite sed -i 's/numpy==1.16.2/numpy/g' requirements.linux.txt > requirements.linux.txt ../../setup.sh

it will take some time to build as numpy needs to get compiled, but then it works. Done on an ubuntu linux server (amd64)

[Dvalin21]

It appears to be possible to make it work.

docker exec -it bash ln -s /usr/include/locale.h /usr/include/xlocale.h cd /app/modules/ObjectDetectionTFLite sed -i 's/numpy==1.16.2/numpy/g' requirements.linux.txt > requirements.linux.txt ../../setup.sh

it will take some time to build as numpy needs to get compiled, but then it works. Done on an ubuntu linux server (amd64)

Will have to give this a try, thanks!!

[rohitksharma]

I am curious if there have been any new updates on CodeProject support for the Coral TPU on non-RPI images?

[douglasfl]

Any updates? will google coral work?

[Dvalin21]

Regrettably I was not able to try this out yet. Had anyone else try it?

[kennethgomez01]

Yes I did able to make this work, typically you need to manually install some dependency package that is not installed from the script. The Module named is "ObjectDetection (TF-Lite)". After serveral time install pycoral.whl and tflite_runtime-2.5.0.post1.whl, I was able to make it work sucessfully. Main issue is that that pycoral is not fully installed or broken when you install the module from codeproject.ai, and usb device coral is not detected.

[garmcqui]

what about if you have a pci/m.2 Coral, will it work?

[XtremeOwnageDotCom]

So...

Blue Iris supports having CodeProject.AI integrated/installed along with it. I have a server, which runs Blue Iris.

My goal, for opening this ticket, is to have the CodeProject.AI which gets installed automatically along with BlueIris, to be able to use a Coral TPU.

That way, I have a self-contained NVR box.

Normally, CodeProject.AI loves to eat up CPU/GPU power for doing detections. Proper Coral TPU support, would allow the TPU to instead handle doing detections.

That being said- quit blowing up this thread with how to get this working on linux, docker, etc.

I already have Frigate running, which works with the Coral TPU effortlessly. My goal here, is to have CodeProject.AI, installed alongside BlueIris, using a Coral TPU. (Without doing funky workarounds, or running a seperate raspberry pi.)

[garmcqui]

So...

Blue Iris supports having CodeProject.AI integrated/installed along with it. I have a server, which runs Blue Iris.

My goal, for opening this ticket, is to have the CodeProject.AI which gets installed automatically along with BlueIris, to be able to use a Coral TPU.

That way, I have a self-contained NVR box.

Normally, CodeProject.AI loves to eat up CPU/GPU power for doing detections. Proper Coral TPU support, would allow the TPU to instead handle doing detections.

That being said- quit blowing up this thread with how to get this working on linux, docker, etc.

I already have Frigate running, which works with the Coral TPU effortlessly. My goal here, is to have CodeProject.AI, installed alongside BlueIris, using a Coral TPU. (Without doing funky workarounds, or running a seperate raspberry pi.)

This works in the newest CPAI release. I have Bi and CPAI installed on windows, and CPAI is using M.2 coral TPU:

[XtremeOwnageDotCom]

This works in the newest CPAI release. I have Bi and CPAI installed on windows, and CPAI is using M.2 coral TPU:

Excellent, let me toss a TPU into my Blue Iris box, and see if I can get this ticket closed out.

[XtremeOwnageDotCom]

Step 1.

Install Coral TPU (I used the M.2 module).

Step 2.

Install drivers https://coral.ai/docs/m2/get-started/#2b-on-windows

After this is completed, you should see TPU in device manager.

Step 3.

In blue iris, turn on codeproject.ai. It should open the download link automatically. Install it.

Step 4. Add Coral Modules.

Step 5. Start it.

That being said, this issue has been completed.

Thanks for the heads up @garmcqui

[Dvalin21]

Can anyone verify that Coral Tpu works outside of Blue Iris other than Raspberry Pi?

[XtremeOwnageDotCom]

Can anyone verify that Coral Tpu works outside of Blue Iris other than Raspberry Pi?

You mean, like it running on windows?

[melyux]

There is no Intel GPU nor Coral TPU support for CPAI on Linux (except Coral, but only on Raspberry Pi). There is support for them when running on Windows. Chris said he's working on both these to be supported on Linux x86_64 (and Docker). Hopefully that will come around, since neither work properly under Windows for me.

[VeganBiker]

Can anyone verify that Coral Tpu works outside of Blue Iris other than Raspberry Pi?

I can confirm that the directions XtremeOwnageDotCom gave work for getting a Coral TPU working on windows 10 as I am using a USB one on a Intel N95 mini PC that is running blue iris.

I can also confirm that the docker pi image works on a non-pi arm ODRIOD running debian bullseye.

I will also note that when I upgraded to CPAI 2.1.10 it broke everything so I downgraded back to 2.1.9 I did however notice that 2.1.11 is now out so I will have to try that. I may wait until I get my upgraded mini PC with a intel i3 N300 CPU

[Dvalin21]

Running CasaOS (docker) on ubuntu server. Getting ready to install my second corral tpu m.2 device into the system just for this purpose.

[melyux]

Running CasaOS (docker) on ubuntu server. Getting ready to install my second corral tpu m.2 device into the system just for this purpose.

You're going to try to run the CPAI image on a non-ARM Ubuntu server and use the Coral? From what is out there, that's not possible (yet... when?), but I would like to be surprised.

[dan-cristian]

Hi all. I have issues starting Coral on Windows 11, I get the error below. Your help is appreciated.

17:16:32:Attempting to start ObjectDetectionCoral with C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\bin\windows\python37\venv\scripts\Python "C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\objectdetection_coral_adapter.py" 17:16:32:Starting C:\Program Files...ws\python37\venv\scripts\Python "C:\Program Files...ectdetection_coral_adapter.py" 17:16:32: 17:16:32:Module 'ObjectDetection (Coral)' 1.5.1 (ID: ObjectDetectionCoral) 17:16:32:Module Path: C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral 17:16:32:AutoStart: True 17:16:32:Queue: objectdetection_queue 17:16:32:Platforms: windows,linux,linux-arm64,macos,macos-arm64 17:16:32:GPU: Support enabled 17:16:32:Parallelism: 1 17:16:32:Accelerator: 17:16:32:Half Precis.: enable 17:16:32:Runtime: python37 17:16:32:Runtime Loc: Local 17:16:32:FilePath: objectdetection_coral_adapter.py 17:16:32:Pre installed: False 17:16:32:Start pause: 1 sec 17:16:32:LogVerbosity: 17:16:32:Valid: True 17:16:32:Environment Variables 17:16:32:MODELS_DIR = %CURRENT_MODULE_PATH%/assets 17:16:32:MODEL_SIZE = large 17:16:32: 17:16:32:Error trying to start ObjectDetection (Coral) (objectdetection_coral_adapter.py) 17:16:32:An error occurred trying to start process 'C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral\bin\windows\python37\venv\scripts\Python' with working directory 'C:\Program Files\CodeProject\AI\modules\ObjectDetectionCoral'. The system cannot find the file specified. 17:16:32: at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo) at CodeProject.AI.Server.Modules.ModuleProcessServices.StartProcess(ModuleConfig module) 17:16:32:Please check the CodeProject.AI installation completed successfully

[laurentopia]

Yes, same. On Win11 Coral turns on and off.

[XtremeOwnageDotCom]

Please go create a new issue for "Coral TPU not working on Windows 11".

This issue is closed, as its original topic, was getting a google coral TPU working at all with codeproject.AI.

Per This comment, The google coral TPU does indeed work with codeproject.AI, on windows 10, and windows server.

Please go create a new issue, this one is closed, and is not going to get much attention....

[Jens-Wymeersch]

@XtremeOwnageDotCom Is ObjectDetection(Coral), the only option you need to activate. Or on top you still need the Object Detection (YOLOv5.6.2) ?

[MikeLud]

@XtremeOwnageDotCom Is ObjectDetection(Coral), the only option you need to activate. Or on top you still need the Object Detection (YOLOv5.6.2) ?

You just need the ObjectDetection(Coral) module.

[Jens-Wymeersch]

@XtremeOwnageDotCom Just to make sure, the coral is the only Object Detection module I need ?

[XtremeOwnageDotCom]

@XtremeOwnageDotCom Just to make sure, the coral is the only Object Detection module I need ?

I am probably not the best person to ask, but, Yes- In my case, only the coral object detection module was needed.

[Jens-Wymeersch]

@XtremeOwnageDotCom how do you know that it is actually working ? I don't see anywhere in the logs objectdetetion_coral_adapter.py : Edge TPU detected or don't you need this in Windows ?

I'm getting at startup

08:36:30:Started ObjectDetection (Coral) module 08:36:37:Module ObjectDetectionCoral has shutdown 08:36:37:objectdetection_coral_adapter.py: has exited

[XtremeOwnageDotCom]

@Jens-Wymeersch I would recommend you open a new issue.