Error while reading sklearn-onnx model

[Vikramardham]

I am testing out the onnxruntime library. I have a very simple model built in Python using sklearn and I save it in the onnx format using skl2onnx library.

python code:

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_digits
from sklearn.linear_model import LogisticRegression
from skl2onnx.common.data_types import FloatTensorType
from skl2onnx import convert_sklearn, get_model_alias

digits = load_digits()
x_train, x_test, y_train, y_test = train_test_split(
    digits.data, digits.target, test_size=0.2, random_state=2)
clf = LogisticRegression()
initial_type = [('input', FloatTensorType([None, 64]))]

onx = convert_sklearn(clf, initial_types=initial_type, target_opset=11)
with open("simple_model.onnx", "wb") as f:
    f.write(onx.SerializeToString()

and I read the model in rust

    let environment = environment::Environment::builder()
        .with_name("trial")
        .with_log_level(onnxruntime::LoggingLevel::Verbose)
        .build()?;

    let _session = environment
        .new_session_builder()?
        .with_optimization_level(onnxruntime::GraphOptimizationLevel::Basic)?
        .with_model_from_file("simple_model.onnx")?

I get the following error while building the session in rust:

thread 'main' panicked at 'assertion failed:(left != right) left:0x0, right:0x0', /home/vikram/.cargo/registry/src/github.com-1ecc6299db9ec823/onnxruntime-0.0.12/src/session.rs:690:9 note: run withRUST_BACKTRACE=1environment variable to display a backtrace

What am I doing wrong? Any help is greatly appreciated!

[harshad-dh]

Running into the same error. Were you able to find any solutions @Vikramardham ? Is sklearn ported model not supported ?

[TimothyOlsson]

I'm getting similar errors

Oct 10 17:30:55.771 DEBUG new{name="test" log_level=Info}: onnxruntime::environment: Environment not yet initialized, creating a new one.
The given version [8] is not supported, only version 1 to 7 is supported in this build.
thread 'main' panicked at 'assertion failed: `(left != right)`
  left: `0x0`,
 right: `0x0`', onnxruntime\src\lib.rs:180:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Seems to originate from here:

fn g_ort() -> sys::OrtApi {
    let mut api_ref = G_ORT_API
        .lock()
        .expect("Failed to acquire lock: another thread panicked?");
    let api_ref_mut: &mut *mut sys::OrtApi = api_ref.get_mut();
    let api_ptr_mut: *mut sys::OrtApi = *api_ref_mut;

    assert_ne!(api_ptr_mut, std::ptr::null_mut());  <-- Here (lib.rs, line 180)

    unsafe { *api_ptr_mut }
}

[tlegrave]

I'm running into a similar issue when trying to load a basic model converted with skl2onnx.

I started investigating and it seems that the error comes from the "deserialization" of the outputs of the model. Sounds like a type casting issue or something similar.

Has anyone found a solution? I'll follow my researchs and post again if I find a workaround.

[tlegrave]

I just found the solution for my case! There are several points to consider when exporting a model with skl2onnx.

Target opset version

Make sure you properly set the target_opset variable. This crate supports onnxruntime V1.8, so make sure you choose a target opset that is supported when you call the function convert_sklearn. You'll find version numbers here.

Output type

The major problem in my case was the output type of the model. By default a sklearn classifier outputs the probabilities in sequence of dictionaries {class: probability_of_the_class}, which is not what onnxruntime expects.

This throws the following error PointerShouldBeNull("TensorInfo") that originates from this line. In fact, the output type can't be bound on a tensor type. To prevent this, you should set the zipmap option to False as mentionned here.

Quick example

Here is a small reproducible example that worked for me:

First, train and export your model in Python:

# Train a classifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

iris = load_iris()
X, y = iris.data, iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y)
clr = DecisionTreeClassifier()
clr.fit(X_train, y_train)

# Export model to ONNX format
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
from skl2onnx.helpers.onnx_helper import save_onnx_model

options = {DecisionTreeClassifier: {'zipmap': False}} # Setting zipmap option to False
initial_type = [('float_input', FloatTensorType([1, 4]))]
model_onnx = convert_sklearn(clr, initial_types=initial_type, target_opset=13, options=options) # Note target_opset version
save_onnx_model(model_onnx, "iris.onnx")

Then load it and run it with Rust:

use onnxruntime::environment::Environment;
use onnxruntime::{GraphOptimizationLevel, LoggingLevel, ndarray};
use onnxruntime::error::Result;
use onnxruntime::tensor::OrtOwnedTensor;

fn main() {
    let environment = Environment::builder()
        .with_name("test")
        .with_log_level(LoggingLevel::Verbose)
        .build().unwrap();

    let mut session = environment
        .new_session_builder().unwrap()
        .with_model_from_file("iris.onnx").unwrap();

    let input0_shape: Vec<usize> = session.inputs[0].dimensions().map(|d| d.unwrap()).collect();
    let output0_shape: Vec<usize> = session.outputs[0]
        .dimensions()
        .map(|d| d.unwrap())
        .collect();

    let array = ndarray::Array1::from(vec![4.9_f32, 3. , 1.4, 0.2]).into_shape(input0_shape);
    let input_tensor = vec![array];

    let outputs: Vec<OrtOwnedTensor<f32,_>> = session.run(input_tensor).unwrap();
    println!("Output : {:?}" ,outputs);
}

Hope this helps!

[Vikramardham]

@tlegrave This is beautiful. Works for me. Thanks a lot!