Weirdly long inference time using cuda? Much much slower

[Jasonzhangzzf]

Environment: python=3.11 Ubuntu 22.04 cuda = 11.8 cudnn = 8.7 onnxruntime-gpu = 1.17 tensor rt = 8.6

1. use_gpu=False
2. use_gpu=True

Is it due to warnings and probably onnxruntime-gpu library version differences?

[Jasonzhangzzf]

The corresponding environment matches what is recommended by the onnxruntime-gpu page.

[jingsongliujing]

Regarding ppocrv4, I have only tested it on CPU. ONNX Runtime may not implement certain operators for CUDA or other reasons.

[Jasonzhangzzf]

Ok then. So CPU inference only. Thanks for the reply!