microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.81k stars 2.94k forks source link

One path in the graph requests feature X(>Y) but input tensor has Y features #16695

Open abilashasomi opened 1 year ago

abilashasomi commented 1 year ago

Describe the issue

Converted xgbclassifier model to ONNX and when inferred via onnxruntime getting below error ERROR:root:[ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running TreeEnsembleClassifier node. Name:'TreeEnsembleClassifier' Status Message: C:\a_work\1\s\onnxruntime\core\providers\cpu\ml\tree_ensemble_common.h:418 onnxruntime::ml::detail::TreeEnsembleCommon<float,float,float>::ComputeAgg One path in the graph requests feature 289 but input tensor has 128 features.

I have debugged and verified the features of model and input data both seems to have 128 only. Also tried below

  1. Change the versions of onnxruntime and onnxmltools
  2. Update the way onnx model is saved.(onnxmltools.utils.save_model/ open a file and write onnx model object as SerializeToString())
  3. Print the model details. In this,found that input dimension is 128 and but somewhere in the 'node_featureids' attribute could find the value 289.

To reproduce

### XGBClassifier to ONNX model conversion after model training:

Define the initial type for input variables

input_type = [('float_input', FloatTensorType([None, <128>]))]

Convert the model to ONNX format

onnx_model = onnxmltools.convert.convert_xgboost(,initial_types=input_type,target_opset=8)

Save the ONNX model

onnx_model_output_path_xgb = os.path.join(args.model_dir, "model.onnx")

onnxmltools.utils.save_model(onnx_model,onnx_model_output_path_xgb)

with open(onnx_model_output_path_xgb, "wb") as f: f.write(onnx_model.SerializeToString())

### ONNX Model Inference code in the Python application:

import onnxruntime as ort

sess = ort.InferenceSession(model_path, providers=ort.get_available_providers()) input_name = sess.get_inputs()[0].name output_name = sess.get_outputs()[0].name output = sess.run([output_name], {input_name: inference_data_np})[0]

ONNX_Model_Details.txt

Urgency

No response

Platform

Linux

OS Version

Ubuntu 22.04

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.15.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

wschin commented 1 year ago

@xadupre, do you have any idea?

xadupre commented 1 year ago

If the highest value is 289, then it is an issue with the converter or xgboost, not onnxruntime. There should be a way to dump a xgboost model and check if the xgboost model contains the feature 289 or not. If yes, then xgboost finds at least 289 features. Otherwise, it is an issue with the converter.

abilashasomi commented 1 year ago

@xadupre @wschin thanks for checking the issue, With the help of Netron app, I have visualized the graph for the onnx model and I could see only 128 features , PFA for the reference, Also please let me know what else information I can share to find the root cause xgbmodel onnx

mvelax commented 1 year ago

A very similar thing is happening to me, narrowed down that this happens on the change from 1.12.1 to 1.13.1.

prithivi1 commented 1 year ago

Hi @abilashasomi @mvelax, Is this issue fixed. I'm facing the same issue !

str8coder commented 7 months ago

I'm experiencing the same issue

ONNX: Non-zero status code returned while running TreeEnsembleClassifier node. Name:'TreeEnsembleClassifier' Status Message: E:\workspace\external\onnx\onnx-runtime\core\providers\cpu\ml\tree_ensemble_common.h:450 onnxruntime::ml::detail::TreeEnsembleCommon<float,float,float>::ComputeAgg One path in the graph requests feature 45 but input tensor has 44 features.

str8coder commented 7 months ago

I was able to resolve the issue by rearranging the dataframe columns. The columns not being used to train, including the label column should be at the end (to the right)