search

snowzach / doods2

API for detecting objects in images and video streams using Tensorflow
MIT License
221 stars 28 forks source link

numThreads and numConcurrent with tflite and Coral usb tpu #87

Closed swedishlars closed 1 year ago

swedishlars commented 1 year ago

Hi, first off, thank you for a very useful, reliable and easy to use tool. Been using Doods for a couple of years now.

Forgive me for opening an issue, technically I do not have one! I'm just trying to understand if numThreads and numConcurrent can help me optimise my setup.

I run doods2 on a Intel Nuc Core i5 with a Coral usb tpu. I cannot find any direct docs about these attributes when using a tpu. Docs states that numThreads can be used with tflite models, but does that work with a usb tpu? Also does numConcurrent make a difference with a usb tpu?

Or should I just set numThreads and numConcurrent to 1 to avoid conflicting threads/instances being sent at the same time to the usb tpu?

Any help is appreciated.

Many thanks!

snowzach commented 1 year ago

I have played around with TPU setting the numThread and concurrent and I'm not sure it made any difference. I think that the TPU itself is single threaded so you can set that but I don't think it will speed it up. I can't honestly tell you for sure as the TPU is kind of a black box.. (well white lulz) Sorry I can't provide more info.

swedishlars commented 1 year ago

Yeah the Google Coral docs hints at it being single, since they talk about adding tpu's for multiple detections. I played around with numThreads and numConcurrent and could not tell a difference. So I will leave both set to 1. Thanks for your reply!

snowzach commented 1 year ago

I think if you had multiple TPUs it automatically load balances them actually...