search

openvinotoolkit / anomalib

An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.
https://anomalib.readthedocs.io/en/latest/
Apache License 2.0
3.73k stars 666 forks source link

[Task]: How to improve OpenVINO Inferece speed? #1225

Closed glucasol closed 1 year ago

glucasol commented 1 year ago

What is the motivation for this task?

I am running this Notebook - 501a_training_a_model_with_cubes_from_a_robotic_arm.ipynb and getting ~16 FPS in inference (CPU: Intel(R) Core(TM) i5-9300H CPU @ 2.40GHz; GPU: NVIDIA GeForce GTX 1650 (dGPU)). I want to know if it's possible to improve it's performance and get more FPS in OpenVINO Inference.

Thanks!

Describe the solution you'd like

I would like to improve the inference performance.

Additional context

No response

samet-akcay commented 1 year ago

Hi @glucasol, we'll get back to you asap! Cheers!

glucasol commented 1 year ago

Thank you @samet-akcay!

samet-akcay commented 1 year ago

@glucasol, one potential approach would be to use NNCF that would accelerate the inference. @paularamo was planning on expanding the notebook with NNCF support, but she is currently out of office. To get an idea you could refer to this NNCF notebook. https://github.com/openvinotoolkit/anomalib/blob/main/notebooks/400_openvino/401_nncf.ipynb

We will hopefully add more to the robotics arm notebook soon. https://github.com/openvinotoolkit/anomalib/blob/main/notebooks/500_use_cases/501_dobot/501a_training_a_model_with_cubes_from_a_robotic_arm.ipynb

samet-akcay commented 1 year ago

Let's continue this from the discussions section. Cheers!