Open Daanfb opened 6 months ago
@Daanfb Hi, I have exact same situation here, have you found any solution yet? Thanks in advance.
@Daanfb @cmc9595 Hi, I have the same problem too. Has there been any progress? Thank you.
First, ensure that there are no onnxruntime or mmdeploy-runtime files on your computer. Delete all such files. Then, install onnxruntime-gpu, mmdeploy, and mmdeploy-runtime-gpu (they cannot coexist).
Install the corresponding versions of TensorRT and cuDNN. After installing ONNX Runtime, everything should work fine. I have installed CUDA 11.8 and ONNX Runtime GPU 11.6.
Additionally,Warning: [W] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading. Solution: set CUDA_MODULE_LOADING=LAZY
@LLH-Harward Thank you for your reply. I also found https://mmdeploy.readthedocs.io/en/latest/04-supported-codebases/mmocr.html#backend-model-inference backend model inference code works fine for me, so I'm trying to combine this code with Triton inference server first. If I fail, I would have to return back to https://mmdeploy.readthedocs.io/en/latest/04-supported-codebases/mmocr.html#sdk-model-inference SDK Inference code using mmdeploy_runtime, so..
@cmc9595 Thank you for your reply! Please let me know if there are any updates.
Checklist
Describe the bug
I have exported the model with this command:
python tools/deploy.py \ configs/mmdet/detection/detection_onnxruntime_dynamic.py \ ../mmpose/projects/rtmpose/rtmdet/person/rtmdet_nano_320-8xb32_coco-person.py \ https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_nano_8xb32-100e_coco-obj365-person-05d8511e.pth \ demo/resources/human-pose.jpg \ --work-dir rtmpose-ort/rtmdet-nano \ --device cpu \ --show \ --dump-info # dump sdk info
In my python code, if i put device='cuda' i get an error, but if I put device='cpu' it works well. How can I use cuda? I tried to execute with --device cuda but I get an error too.
The route indicated (rtmpose-ort/rtmdet-nano) is a directory with the .onnx and .json files
Reproduction
from mmdeploy_runtime import Detector
det_model_path = 'rtmpose-ort/rtmdet-nano' device_name = 'cuda'
bbox_detector = Detector(det_model_path, device_name, device_id=0)
Environment
Error traceback