how to enhance the model‘s performence when running on cpu ？

[kyriewentop]

If there are some methods I can take to accelerate the inference speed by cpu ? I got a task which is to make it feasible that the model's inference speed is above 15fps without gpu . It bothers me a lot , I guess I need some help.

[amcones]

I have tested on RTX 3060Ti and intel i7-7700K, the model's inference speed is about 30fps and 2fps. It is a far cry from the offical documents.

[kyriewentop]

I have tested on RTX 3060Ti and intel i7-7700K, the model's inference speed is about 30fps and 2fps. It is a far cry from the offical documents.

I got the similar results to yours.

[PFC-star]

Do you deploy it with cpp?

[Xzavierlovescoding]

cpp+onnx推理，我的结果也一样2fps左右

[cvDFT]

如果有一些方法，我可以采取cpu来加速推理速度？我得到了一个任务，即使模型的推理速度在没有gpu的情况下高于15fps。这让我很困扰，我想我需要一些帮助。

尝试框架

[yuqilol]

openvino加速cpu最快，能有个3-5倍

[YiQiu1984]

I have the same question , I want to deploy it on windows with cpu inference.