Open papago2355 opened 9 months ago
Hello, this certainly isn't expected behavior. Just to clarify, in your config you use 512x512 resolution which you say takes 1.2s to execute, which resolution takes 15s? In any case, EfficientAD should be fast, but it does use neural networks, so if you want the execution to be fast, you will need to run it on a GPU. I suspect this is the main culprit. Also make sure you are timing only the inference (predict time), not the model instantiation (and other parts), as that would falsely inflate the duration (unless your task involves this).
Hi, thanks for help. When I use 700x1200 it takes 15s to get prediction. I only check during inference so instantiation model does not be a reason for too much time. Is it normal it takes way more times when use openvino? Other than EfficientAD, it usually takes less time to get predictions with openvino inferencer.
Okay, as you mentioned, it is maybe due to use openvino in cpu option. So I tested torch_inferencer, now it is really faster. with 700x1200, I got 0.6 for prediction return. But still not sure why it works slow in openvino inferencer.
Did you also try openvino with gpu inference? You can pass device parameter and set it to GPU by using --device GPU
.
The reason why the CPU is so much slower (in most cases) than GPU is that neural networks can be highly parallelized, benefiting from GPU parallel execution significantly.
@blaz-r, do you think if this is a bug or a specific use-case?
@samet-akcay I don't think this is a bug, but it surely is a bit unusual for EfficientAD to take so much time. Maybe the reason is inference on the CPU or it could also be something with OpenVINO but I can't say for sure.
@samet-akcay Hi. I recentely switched inference module to TorchInferencer and it seems like the issue has been addressed. However, still does not know why it gets extremely slower when I use OpenvinoInferencer(It gives same results. No accuracy drop nor incorrect prediction. Just prediction speed gets 10x slower). Maybe it is due to huge image size?(since I am using 800x1600 image size).
Describe the bug
I am using Openvino inference to predict anomalies in custom datasets. However, it takes too much times compared to other models like Padim or FastFlow. In same width/height, Padim takes 0.33 s and Fastflow takes 0.8. However, Efficient_AD takes 15s.
Is it normal that Openvino export+EfficientAD takes more times to predict compared to padim or fastflow?
Dataset
Folder
Model
Other (please specify in the field below)
Steps to reproduce the behavior
OS information
OS information:
Expected behavior
I was expected Efficient_AD has better speed compared to Padim and Fastflow. Also, with 512x512, Efficient_AD with openvino inferencer takes 1.2s. Padim takes 0.2s.
Screenshots
No response
Pip/GitHub
pip
What version/branch did you use?
No response
Configuration YAML
Logs
Code of Conduct