[ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for Where(9) node with name '/model.24/transformer_cross_attention_layers.0/Where_1'

[JiayuanWang-JW]

Describe the issue

I install the 1.19.0 onnxruntime. I think it should be supported Where operator. However, I can not run it and got an error: onnxruntime.capi.onnxruntime_pybind11_state.NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for Where(9) node with name '/model.24/transformer_cross_attention_layers.0/Where_1'

To reproduce

I based on YOLOv8 to build the model. However, I add the transformer modules here. I converted to onnx format successfully, but I can not use it. I have tried to change the opset version from 12 to 21. But nothing.

Urgency

No response

Platform

Linux

OS Version

22.04.2

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

ONNX Runtime Version (1.19.0)

ONNX Runtime API

Python

Architecture

X64

Execution Provider

CUDA

Execution Provider Library Version

CUDA 12.1

[tianleiwu]

@JiayuanWang-JW,

Where operator is supported by onnxruntime and it is common in transformer model to use Where for padding mask. It is likely that your onnx model is not valid.

You can use https://onnx.ai/onnx/api/checker.html to verify onnx model is valid. Or shall your model for investigation.

[JiayuanWang-JW]

@JiayuanWang-JW,

Where operator is supported by onnxruntime and it is common in transformer model to use Where for padding mask. It is likely that your onnx model is not valid.

You can use https://onnx.ai/onnx/api/checker.html to verify onnx model is valid. Or shall your model for investigation.

Hi @tianleiwu,

I try through the below code to check my onnx model.

import onnx
import onnx.checker

model_path = "/home/jiayuan/best.onnx"
model = onnx.load(model_path)

try:
    onnx.checker.check_model(model, full_check=True)
    print("The model is valid!")
except onnx.checker.ValidationError as e:
    print(f"The model is invalid: {e}")

I get the

The model is valid!

Process finished with exit code 0

Please let me know if I did any wrong operation.

I hope I can share the onnx model. However, the model is large, like 125.75 MB. How can I share this with you?

[tianleiwu]

@JiayuanWang-JW, you can save the model like the following, and attach the model.onnx only (not need model.onnx.data).

onnx.save_model(model, "model.onnx", save_as_external_data=True, all_tensors_to_one_file=True, location="model.onnx.data", size_threshold=128, convert_attribute=False)

[tianleiwu]

For CUDA, Where operator only supports uint8, int32, int64, float, double, float16, bfloat16. It seems that the model uses bool as X type. It is valid for ONNX, however not implemented in ORT.

Could you modify the modeling script (the transformer part, add some data type conversion to use supported type)?

[JiayuanWang-JW]

For CUDA, Where operator only supports uint8, int32, int64, float, double, float16, bfloat16. It seems that the model uses bool as X type. It is valid for ONNX, however not implemented in ORT.

Could you modify the modeling script (the transformer part, add some data type conversion to use supported type)?

Thanks for the clarification.

I will try your suggestion.