DirectML returning empty result with ObjectDetection (Mobilinet V2 FPN Keras)

[willianwrm]

Describe the bug Result is empty when using DirectML but works as expected when using only CPU. Tested for ObjectDection with ssd_mobilenet_v2_fpn_keras model.

Urgency low

System information

• OS Platform and Distribution: Windows 11 23H2 22631.3447
• ONNX version: 1.17.3
• GPU: NVIDIA RTX 3060Ti driver version 31.0.15.5212 and Intel UHD 730 driver version 31.0.101.5084

System information for training and ONNX model conversion

• OS Platform and Distribution: Linux Ubuntu 22.04.3 LTS in WSL2
• TensorFlow: 2.15
• ONNX version: 1.16.1, tested with GIT version as well
• piplist.txt

To Reproduce Use this C# .NET 8.0 Windows Forms App, select a file image and wait for the result, the CPU result is printed in blue and DirectML result in red. The project contains the onnx model and image examples can be found here. Tested opset 18 to 13, no change.

Screenshots

Additional context Training configuration, data set, command lines and tensorflow exported graph can be found here

[willianwrm]

Update: i did force the _numdetections to 10 just to see what values I would get and surprising got good _detectionboxes but bad _detectionscores (-1 for all):

So investigating a little further i found out that the _outputs detection_anchorindices, _detectionboxes, _detectionclasses, _detection_multiclassscores, _raw_detectionboxes and even _raw_detectionscores are returning correct values (DirectML outputs same as CPU), only the _detectionscores and _numdetections that are messed up.

[fatcat-z]

Do you mean the model could be converted to ONNX model successfully and it works well on CPU but failed in other scenarios?

If so, this issue probably should be opened in onnxruntime repo because it looks like an inference issue.

[willianwrm]

Yes it does works well on CPU but fails in DirectML. Thanks for the info, i did post it there.