microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.58k stars 2.92k forks source link

Model saved by ORT as external data format will not be aligned for mapfile support #21524

Open ivberg opened 3 months ago

ivberg commented 3 months ago

Describe the issue

ORT has the ability to save a model with external data format - https://github.com/onnx/onnx/blob/main/docs/ExternalData.md

However, this model will not be aligned and mapfile support will not work - #21195

To reproduce

session_options.SetGraphOptimizationLevel(ORT_DISABLE_ALL); // ORT_DISABLE_ALL, ORT_ENABLE_BASIC, ORT_ENABLE_EXTENDED, ORT_ENABLE_ALL: // Ideally the model would be saved with basic or higher optimization level. However, there is a related bug on this OnnxRuntime 21325 -Model saved with offline basic optimizations will not load – ShapeInferenceError

    session_options.SetOptimizedModelFilePath(optimized_model_path);
    session_options.AddConfigEntry(kOrtSessionOptionsOptimizedModelExternalInitializersFileName, "model.quant.extdata.disable_opt.onnx.data");
    session_options.AddConfigEntry(kOrtSessionOptionsOptimizedModelExternalInitializersMinSizeInBytes, "10");

    session = Ort::Session(env, filemodelpath_base_unencrypted, session_options);

Urgency

Low urgency. There is the ability to add alignment in Python in PR - https://github.com/onnx/onnx/pull/6248

Platform

Windows

OS Version

11

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.18.1

ONNX Runtime API

C++

Architecture

ARM64

Execution Provider

Default CPU

Execution Provider Library Version

No response

github-actions[bot] commented 2 months ago

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.