onnx offline demo在gpu上运行非常慢，比cpu慢了差不多5倍

[Ignalxy]

onnx offline demo在gpu上运行非常慢，比cpu慢了差不多5倍

运行pytorch的demo，gpu比cpu快，很正常，但是 运行onnx的demo，device=0推理一段120s的音频，比在cpu上慢了5倍 运行c++版本的也是一样，甚至比python的demo还慢

Code

from funasr_onnx import Paraformer
from pathlib import Path

model_dir = "damo/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch"
# model_dir = "damo/speech_paraformer-large-vad-punc_asr_nat-zh-cn-16k-common-vocab8404-pytorch"
model = Paraformer(model_dir, batch_size=1, quantize=False)
# model = Paraformer(model_dir, batch_size=1, device_id=0)  # gpu

# when using paraformer-large-vad-punc model, you can set plot_timestamp_to="./xx.png" to get figure of alignment besides timestamps
# model = Paraformer(model_dir, batch_size=1, plot_timestamp_to="test.png")

wav_path = ['/work/lxy/audio/segment_0-120.wav']

result = model(wav_path)
print(result)

What's your environment?

• Ubuntu 22.04
• funasr: 1.0.26
• funasr-onnx 0.3.1
• ModelScope: 1.14.0
• PyTorch Version:2.3.0
• How you installed funasr (pip, source): pip
• Python version: 3.9.19
• GPU: A30
• CUDA/cuDNN version (e.g., cuda11.7): 12.3

[lyblsgo]

onnx模型在CPU上的优化做得好，GPU上不太行，不建议把onnx的模型部署在GPU上 我们最近会推出一版基于torchscript的GPU的服务部署，到时可以再试一下