Closed prikmm closed 2 years ago
This is currently a limitation of the implementation. see https://github.com/microsoft/onnxruntime/pull/5840 I'll provide some background context. OnnxRuntime Execution Providers can be implemented using one of two approaches. The classic approach is to implement kernels which follow the corresponding ONNX operator specification. The implementation signatures are registered in an EP’s kernel registry. These signatures are static and known at build time. At runtime, OnnxRuntime assigns nodes in the graph to an EP when it can find a matching signature in that EP’s kernel registry. The CUDA Execution Provider follows this pattern of implementation. The other approach is to implement the Function Kernel (aka Compiled Kernel) interface, where the EP produces a list of subgraphs it supports, and the framework rewrites the main graph, fusing each subgraph into a single node. A matching implementation signature is generated by the framework at runtime and used for assigning the fused nodes to the associated EP. TensorRT, OpenVINO, and oneDNN Execution Providers implement the Function/Compiled Kernel approach. We don't support serializing graphs which contain fused Execution Provider nodes. The signatures won't exist in the static registry and the model will fail to load (model is considered invalid).
Closing as this is by design
Describe the bug A clear and concise description of what the bug is. To avoid repetition please make sure this is not one of the known issues mentioned on the respective release page.
I am trying to save the optimized model using
optimized_model_filepath
inSessionOptions
after the model has been loaded and optimized using the execution providers in the inference session but I keep getting the below displayed error:I don't get this error when I use CUDA. But, when I use oneDNN, OpenVINO and TensorRT Execution Providers this happens.
Urgency If there are particular important use cases blocked by this or strict project-related timelines, please share more information and dates. If there are no hard deadlines, please specify none.
System information
To Reproduce
sess_options = ort.SessionOptions() sess_options.optimized_model_filepath =
session = ort.InferenceSession(,
providers=[],
sess_options=sess_options)