microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.12k stars 2.84k forks source link

Why C++ cannot modify the enable_mem_reuse option in Ort::SessionOptions... #21942

Open WYX523 opened 2 weeks ago

WYX523 commented 2 weeks ago

Describe the issue

I have trained a model and export to onnx, but I got an error when infer by ort: Non-zero status code returned while running Expand node. Name:'/GaussianDiffusionModel/Expand' Status Message: /GaussianDiffusionModel/Expand: left operand cannot broadcast on dim 0 LeftShape: {110}, RightShape: {118}

when I unable the enable_mem_reuse option in python, it works, but I cannot find the modification method in cpp, how can I unable this option with c/cpp program and resolve this problem?

To reproduce

inline std::shared_ptr<Ort::SessionOptions> CreateDefaultSessionOptions(size_t numThreads) {
    auto sessionOptionsPtr = std::make_shared<Ort::SessionOptions>();
    sessionOptionsPtr->SetIntraOpNumThreads(numThreads);
    sessionOptionsPtr->SetInterOpNumThreads(numThreads);
    sessionOptionsPtr->SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_DISABLE_ALL);
    return sessionOptionsPtr;
}

Urgency

No response

Platform

Linux

OS Version

CentOS

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.18.1

ONNX Runtime API

C++

Architecture

X86

Execution Provider

Default CPU, CUDA

Execution Provider Library Version

CUDA 11.8

WYX523 commented 2 weeks ago

And I'm so doubtful that the option is opened in python but not in c++...

pranavsharma commented 1 week ago

Not sure what you mean by "unable". Here are the C++ equivalents: EnableCpuMemArena() and DisableCpuMemArena().