AkkiSony
commented
2 years ago @manoj7410 Thank for your input. I will try that! Is there a way where I can compare few parameters with other harware accelerators? What parameters would you try to compare if you would be using two different accelerators? @hjonnala
Description
I am able to run the model on Edge TPU, however I just have small question with respect to the inference time.
----INFERENCE TIME---- Note: The first inference is slow because it includes loading the model into Edge TPU memory. 154.32 ms 24.17 ms 23.97 ms 21.98 ms 27.74 ms
The first inference is with loading the model and the rest of the inference time is slow as the model is already loaded into the edge tpu right?
I came across the snippet which calculates the inference time:
for _ in range(args.count): start = time.perf_counter() interpreter.invoke() inference_time = time.perf_counter() - start objs = detect.get_objects(interpreter, args.threshold, scale) print('%.2f ms' % (inference_time * 1000))
My question is how does the code avoid to load the model from the second loop onwards?
Just a small simple query I wanted to get clarified. Thanks in advance! :)