microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.88k stars 2.95k forks source link

CenterCropPad inside of SequenceMap subgraph produces INVALID_GRAPH #22100

Open DRKV333 opened 2 months ago

DRKV333 commented 2 months ago

Describe the issue

I'm trying to use a model with onnxruntime that does some preprocessing on a sequence of input images. Among other nodes, I have a CenterCropPad inside of a subgraph, that is the body of a SequenceMap. This however causes the constructor of onnxruntime.InferenceSession to throw an exception.

To reproduce

import onnx
import onnxruntime

repro = onnx.parser.parse_model("""
<
    ir_version: 8,
    opset_import: [ "" : 21 ]
>
preprocess (seq(float[X, Y]) images) => (float[N, 5, 5] preprocessed)
{
    seq = SequenceMap<
        body=preprocess_single(float[X, Y] image) => (float[5, 5] cropped)
        {
            size = Constant<value=int64[2] {5, 5}>()
            cropped = CenterCropPad<axes=[0, 1]>(image, size)
        }
    >(images)
    preprocessed = ConcatFromSequence<axis=0, new_axis=1>(seq)
}
""")

onnx.checker.check_model(repro)
onnx.save(repro, "/home/jupyter/repro.onnx")

session = onnxruntime.InferenceSession("/home/jupyter/repro.onnx", None)
---------------------------------------------------------------------------
InvalidGraph                              Traceback (most recent call last)
Cell In[193], line 1
----> 1 session = onnxruntime.InferenceSession("/home/jupyter/repro.onnx", None)

File ~/.local/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419, in InferenceSession.__init__(self, path_or_bytes, sess_options, providers, provider_options, **kwargs)
    416 disabled_optimizers = kwargs.get("disabled_optimizers")
    418 try:
--> 419     self._create_inference_session(providers, provider_options, disabled_optimizers)
    420 except (ValueError, RuntimeError) as e:
    421     if self._enable_fallback:

File ~/.local/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:491, in InferenceSession._create_inference_session(self, providers, provider_options, disabled_optimizers)
    488     disabled_optimizers = set(disabled_optimizers)
    490 # initialize the C++ InferenceSession
--> 491 sess.initialize_session(providers, provider_options, disabled_optimizers)
    493 self._sess = sess
    494 self._sess_options = self._sess.session_options

InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : This is an invalid model. In Node, ("", Loop, "", -1) : ("_inlfunc_SequenceMap_SequenceMap_input_sequence_seqlen": tensor(int64),"_inlfunc_SequenceMap_SequenceMap_input_sequence_cond": tensor(bool),"_inlfunc_SequenceMap_SequenceMap_out_sequence_0_seqempty": seq(tensor(float)),) -> ("seq": seq(tensor(float)),) , Error Nodes in a graph must be topologically sorted, however input '_inlfunc_CenterCropPad_padded_input' of node: 
name:  OpType: Slice
 is not output of any previous nodes.

If I try to just run the subgraph on its own, everything works fine:

<
    ir_version: 8,
    opset_import: [ "" : 21 ]
>
preprocess_single(float[X, Y] image) => (float[5, 5] cropped)
{
    size = Constant<value=int64[2] {5, 5}>()
    cropped = CenterCropPad<axes=[0, 1]>(image, size)
}

If I switch the crop to a resize, that also works:

<
    ir_version: 8,
    opset_import: [ "" : 21 ]
>
preprocess (seq(float[X, Y]) images) => (float[N, 5, 5] preprocessed)
{
    seq = SequenceMap<
        body=preprocess_single(float[X, Y] image) => (float[5, 5] cropped)
        {
            size = Constant<value=int64[2] {5, 5}>()

            cropped = Resize<
                mode=\"linear\",
                axes=[0, 1]
            >(image, , , size)
        }
    >(images)
    preprocessed = ConcatFromSequence<axis=0, new_axis=1>(seq)
}

Urgency

Affected project is for my own amusement, so it's not particularly urgent.

Platform

Linux

OS Version

Debian GNU/Linux 12 (bookworm)

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.19.2

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.