Error: Type parameter (T) of Optype (Where) bound to different types (tensor(float) and tensor(double) in node (/model/decoder/Where).

[ramonhollands]

Star RTDETR 请先在RTDETR主页点击star以支持本项目 Star RTDETR to help more people discover this project.

---

Describe the bug Trying to run inference on the onnx file gives an error: Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /content/RT-DETR/rtdetr_pytorch/model.onnx failed:Type Error: Type parameter (T) of Optype (Where) bound to different types (tensor(float) and tensor(double) in node (/model/decoder/Where).

To Reproduce https://colab.research.google.com/drive/1Ievc9FkjJALoWBjOo4Q6f0wu5buKIZiy?usp=sharing

[verrassendhollands]

I tried it in a different colab and the error disappeared. So it might be an issue related to the environment? https://colab.research.google.com/drive/1Gh2YH_ojOoLOI6ExGupoeR8fRaNoHiBW?usp=sharing

I discovered however that Yolov5 runs approx 3 times (!) faster (via onnxrunner - cpu). I'll try it with scaling and reducing the number of decoders. Do you think that will have that much impact?

[lyuwenyu]

@ramonhollands

Do you use the latest code? please check these two lines that using where op

1. https://github.com/lyuwenyu/RT-DETR/blob/main/rtdetr_pytorch/src/zoo/rtdetr/rtdetr_decoder.py#L484
2. https://github.com/lyuwenyu/RT-DETR/blob/main/rtdetr_pytorch/src/zoo/rtdetr/rtdetr_decoder.py#L466

[ramonhollands]

Thanks for your reply! Yes, I'm using the latest code

[ramonhollands]

I have fixed it by replacing line 466 from anchors = torch.where(valid_mask, anchors, torch.inf) to anchors = torch.where(valid_mask, anchors, torch.tensor(float('inf'), dtype=torch.float32, device=device))

[ramonhollands]

About the speed issue: Yolov5 runs in about 54ms on my machine (CPU) RTDETR scaled 18 with 2 decoders runs in about 121ms on my machine (CPU)

I would expect RT-DETR to be much faster.

I'm using these configs: PResNet: depth: 18 freeze_at: -1 freeze_norm: False pretrained: True

HybridEncoder: in_channels: [128, 256, 512] hidden_dim: 256 expansion: 0.5 depth_mult: 0.33

RTDETRTransformer: eval_idx: 2 num_decoder_layers: 3 num_denoising: 100

[lyuwenyu]

We test end-to-end speed on GPU with tensorrt-fp16.

[verrassendhollands]

@lyuwenyu Thanks for your answers. Just one final check. It is correct that the exported onnx file is about 80MB? (r18vd)