Performance in CLI vs Service vs Docker 2x, 4x reduction

robmarkcole / tensorflow-lite-rest-server

Expose tensorflow-lite models via a rest API using FastAPI

Apache License 2.0

77 stars 21 forks source link

Closed fgonza2 closed 2 years ago

fgonza2 commented 3 years ago

I am getting <200ms while i run it in CLI. If i run it as a service it goes up to 500ms. then in docker goes up to 800ms

Running Ubuntu on a 7th Gen Intel CPU with Coral USB installed

Any ideas why?

robmarkcole commented 3 years ago

That is uesful to know, but your guesses are as good as mine