Closed deepakkupanda closed 1 year ago
No one has been assigned till now. Please investigate the issue. Thanks
Sorry for late reply, I can't reproduce your problem with docker image.
When convert the model, there will be two visualized images. Does it looks normal?
# docker run -it --rm --gpus all -v ~/home:/home --network host mmdeploy:cuda
# under workspace
export LD_LIBRARY_PATH=/root/workspace/onnxruntime-linux-x64-1.8.1/lib:$LD_LIBRARY_PATH
python mmdeploy/tools/deploy.py mmdeploy/configs/mmdet/detection/detection_onnxruntime_dynamic.py mmdetection/configs/ssd/ssd512_coco.py https://download.openmmlab.com/mmdetection/v2.0/ssd/ssd512_coco/ssd512_coco_20210803_022849-0a47a1ca.pth mmdetection/demo/demo.jpg --work-dir mmdeploy_model/ssd512_coco_fp16_dynamic --device cuda --dump-info
# then in /root/workspace/mmdeploy_model/ssd512_coco_fp16_dynamic
# there will be two visualized image.
# below is mine.
The conversion worked for me. I am using my own python script to process the bounding box. The same python file gives output for DETR but does not work for SSD. Please let me know if the bounding box returned by DETR and SSD512 are different? I mean, is bounding box are returning x,y,x,y format or something else. Please let me know.
One more thing is around the inference time. Usually ssd takes 30-40ms but here converted onnx is taking 330ms. Any reason for this?
The inference time quoted is in cpu. The pytorch version takes 30-40 ms but the onnx version takes ten times more. Why is it so?
The return format of dets for detection is same (left, top, right, bottom, score)
When you convert a model, there will be two visualized image, one is inferenced by pytorch and the other is inferenced by backend model. If the result of backend model looks reasonable, the model is likely to be fine. You said the conversion worked for you, did you mean that the visualized result of onnxruntime looked reasonable?
I test ssd512_coco.py
on my pc and the result looks fine. The inference time is about 26ms (average for 100 inference times). You said you used same script for DETR
and SSD
, while according to https://github.com/open-mmlab/mmdetection/blob/master/configs/detr/detr_r50_8x2_150e_coco.py#L72, the std
for these two model are not same, the empty ssd output may be caused by wrong std
in your scripts?
Below is my test script and visualized result.
import onnxruntime as ort
import cv2
import numpy as np
import time
model = '/root/workspace/mmdeploy_model/ssd512_coco_fp16_dynamic/end2end.onnx'
imgpath = '/root/workspace/mmdetection/demo/demo.jpg'
sess= ort.InferenceSession(model)
mean = np.array([123.675, 116.28, 103.53]).astype(np.float32)
std = np.array([1, 1, 1]).astype(np.float32)
img = cv2.imread(imgpath)
img = cv2.resize(img, (512, 512)).astype(np.float32)
img = img[:, :, ::-1]
img = (img - mean) / std
img = np.transpose(img, (2, 0, 1))
img = img[None]
input_feed = {'input': img}
thres = 0.2
dets, labels = sess.run(['dets', 'labels'], input_feed=input_feed)
img = cv2.imread(imgpath)
for i in range(dets.shape[1]):
l, t, r, b, s = dets[0, i]
if s < thres:
break
scaleh = 512. / img.shape[0]
scalew = 512. / img.shape[1]
l = l / scalew
r = r / scalew
t = t / scaleh
b = b / scaleh
img = cv2.rectangle(img, (int(l), int(t)), (int(r), int(b)), (0,0,255))
cv2.imwrite('img.jpg', img)
# test inference time
def test_inference_time():
import time
st = time.perf_counter()
loop = 100
for i in range(loop):
dets, labels = sess.run(['dets', 'labels'], input_feed=input_feed)
ed = time.perf_counter()
print((ed - st) / loop)
test_inference_time()
# 0.026346996149995902
Checklist
Describe the bug
ssd300 and ssd512 do not output any bounding box. I have converted the checkpoint to onnx using this config file "mmdeploy/configs/mmdet/detection/detection_onnxruntime_dynamic.py". The onnx model is successful but when testing the onnx model it does not output any bounding box. I also tried converting the checkpoint using the mmdetection onnx converstion script as well. However, I did not get any result.
Reproduction
python mmdeploy/tools/deploy.py mmdeploy/configs/mmdet/detection/detection_onnxruntime_dynamic.py mmdetection/configs/ssd/ssd512_coco.py mmdetection/checkpoints/ssd512_coco_20210803_022849-0a47a1ca.pth mmdetection/demo/demo.jpg --work-dir mmdeploy_model/ssd512_coco_fp16_dynamic --device cuda --dump-info
Environment
Error traceback