Onnxruntime error during inference with operator (CumSum)

[TheMattBin]

Describe the issue

After I export the onnx model with huggingface optimum, I run the following script to check my model which has no issue.

import onnx
onnx_model = onnx.load("/content/deta_test/model.onnx")
onnx.checker.check_model(onnx_model)

However, when I tried to run inference with onnxruntime, I got the following error:

InvalidGraph                              Traceback (most recent call last)
[<ipython-input-7-53b3f77fe0ed>](https://localhost:8080/#) in <cell line: 11>()
      9 # Step 2: Load the ONNX model
     10 model_path = "/content/deta_test/model.onnx"  # Update with your model path
---> 11 ort_session = ort.InferenceSession(model_path)
     12 
     13 # Step 3: Prepare inputs for inference

1 frames
[/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py](https://localhost:8080/#) in _create_inference_session(self, providers, provider_options, disabled_optimizers)
    478 
    479         if self._model_path:
--> 480             sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
    481         else:
    482             sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)

InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : Load model from /content/deta_test/model.onnx failed:This is an invalid model. Type Error: Type 'tensor(bool)' of input parameter (/model/Equal_7_output_0) of operator (CumSum) in node (/model/CumSum_1) is invalid.

To reproduce

Environment: Google Colab

Inference code:

import numpy as np
import onnxruntime as ort

batch_size = 1 
pixel_values = np.random.rand(batch_size, 3, 800, 1066).astype(np.float32)
pixel_mask = np.random.randint(0, 2, size=(batch_size, 800, 1066)).astype(np.int64)

model_path = "/content/deta_test/model.onnx"  # Update with your model path
ort_session = ort.InferenceSession(model_path)

input_names = [input.name for input in ort_session.get_inputs()]
inputs = {
    input_names[0]: pixel_values,
    input_names[1]: pixel_mask
}

outputs = ort_session.run(None, inputs)

print(outputs) 

Urgency

No response

Platform

Web Browser

OS Version

Google Colab

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.19.2

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

[gramalingam]

Looks like a type error in the model. Do you have any instructions for exporting the model, to repro this? Looks like somebody is trying to do a cumulative-sum over boolean values produced using the Equality operator, but CumSum doesn't support boolean. So, I guess a cast from boolean to Int32 or Int64 may be required. Could be an exporter/converter bug.

[TheMattBin]

Hi, thanks for the reply! I actually use Huggingface optimum to export my onnx model, below would be the script,

Script here

```python from optimum.exporters.onnx import main_export from transformers import AutoConfig from optimum.exporters.onnx.model_configs import ViTOnnxConfig from optimum.utils import DummyVisionInputGenerator, DEFAULT_DUMMY_SHAPES, NormalizedVisionConfig from typing import Dict class DetaDummyInputGenerator(DummyVisionInputGenerator): def __init__( self, task: str, normalized_config: NormalizedVisionConfig, batch_size: int = DEFAULT_DUMMY_SHAPES["batch_size"], num_channels: int = DEFAULT_DUMMY_SHAPES["num_channels"], width: int = DEFAULT_DUMMY_SHAPES["width"], height: int = DEFAULT_DUMMY_SHAPES["height"], **kwargs, ): super().__init__( task=task, normalized_config=normalized_config, batch_size=batch_size, num_channels=num_channels, width=width, height=height, **kwargs, ) self.width = 1066 self.height = 800 def generate(self, input_name: str, framework: str = "pt", int_dtype: str = "int64", float_dtype: str = "fp32"): input_ = super().generate( input_name=input_name, framework=framework, int_dtype=int_dtype, float_dtype=float_dtype ) return input_ class DetaOnnxConfig(ViTOnnxConfig): DEFAULT_ONNX_OPSET = 16 DUMMY_INPUT_GENERATOR_CLASSES = (DetaDummyInputGenerator, ) ATOL_FOR_VALIDATION = 1e-3 @property def inputs(self) -> Dict[str, Dict[int, str]]: return { "pixel_values": {0: "batch_size"}, 'pixel_mask': {0: "batch_size"} } model_id = "jozhang97/deta-resnet-50" config = AutoConfig.from_pretrained(model_id, trust_remote_code=True) onnx_config = DetaOnnxConfig( config=config, task="object-detection" )