[Bug]: Onnxruntime.CPU memoty leaks

EjenY-Poltavchiny commented 1 month ago

Describe the issue

Hi, there! I've started to work with onnxruntime lib aproximately 2 mounth ago. I've tried to search for documentation for C++ and found some inferencing examples from ultralitycs and others (including your examples). After all i succesfuly run model inferencing and spent some time for postprocessing. However, I decided to check my code for memory leaks and other problems using GCC sanitizer and found some memory leaks. After endless attempts to fix it, I still didn't understand why my code causes memory leaks, so i decided to check docs examples for memory leaks. And GCC sanitizer found it:

==3972466==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7fc26e8d5a57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7fc26e3b4df9  (/usr/local/lib/libonnxruntime.so.1.18.0+0xa2cdf9)

Direct leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7fc26e8d5a57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7fc26e3b4ddd  (/usr/local/lib/libonnxruntime.so.1.18.0+0xa2cddd)

I've used this example.

They all will cause the same leak with no explainings (i've tried to find symbols by adress that sanitizer gives me, but it was no use)

==3991181==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7f353d9ffa57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f353d4dedf9  (/usr/local/lib/libonnxruntime.so.1.18.0+0xa2cdf9)

Direct leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7f353d9ffa57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f353d4deddd  (/usr/local/lib/libonnxruntime.so.1.18.0+0xa2cddd)

SUMMARY: AddressSanitizer: 256 byte(s) leaked in 2 allocation(s).

To reproduce

This was my Cmake:

cmake_minimum_required(VERSION 3.5)

set(PROJECT_NAME Yolov8OnnxRuntimeCPPInference)
project(${PROJECT_NAME} VERSION 0.0.1 LANGUAGES CXX)

add_executable(Yolov8OnnxRuntimeCPPInference main.cpp)
target_link_libraries(Yolov8OnnxRuntimeCPPInference PRIVATE onnxruntime)

target_compile_options(Yolov8OnnxRuntimeCPPInference PRIVATE -fsanitize=address)
target_link_options(Yolov8OnnxRuntimeCPPInference PRIVATE -fsanitize=address)

If i run this code it will cause memory leaks:

int main() {
  // onnxruntime setup
  Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "example-model-explorer");
  return 0;

Or this one:

int main() {
  // onnxruntime setup
  for (int i = 0; i < 10; ++i) {
    Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "example-model-explorer");
  }
  return 0;

With ASAN_OPTIONS=fast_unwind_on_malloc=0:

==2926944==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7f7139f80a57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f7139a60709  (/lib/libonnxruntime.so.1.18.1+0xa2d709)
    #2 0x7f7138a454de in __pthread_once_slow /build/glibc-wuryBv/glibc-2.31/nptl/pthread_once.c:116
    #3 0x7f71392f5774  (/lib/libonnxruntime.so.1.18.1+0x2c2774)
    #4 0x7f71392f6561  (/lib/libonnxruntime.so.1.18.1+0x2c3561)
    #5 0x7f7139291a4c  (/lib/libonnxruntime.so.1.18.1+0x25ea4c)
    #6 0x7f7139268e34  (/lib/libonnxruntime.so.1.18.1+0x235e34)
    #7 0x55e7b70402ae in Ort::Env::Env(OrtLoggingLevel, char const*) /usr/include/onnxruntime_cxx_inline.h:481
    #8 0x55e7b703f820 in main /home/ejen/Downloads/onnxruntime_ex/main.cpp:228
    #9 0x7f7138a83082 in __libc_start_main ../csu/libc-start.c:308
    #10 0x55e7b703f62d in _start (/home/ejen/Downloads/onnxruntime_ex/build/Yolov8OnnxRuntimeCPPInference+0x262d)

Direct leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7f7139f80a57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7f7139a606ed  (/lib/libonnxruntime.so.1.18.1+0xa2d6ed)
    #2 0x7f7138a454de in __pthread_once_slow /build/glibc-wuryBv/glibc-2.31/nptl/pthread_once.c:116
    #3 0x7f71392f5774  (/lib/libonnxruntime.so.1.18.1+0x2c2774)
    #4 0x7f71392f6561  (/lib/libonnxruntime.so.1.18.1+0x2c3561)
    #5 0x7f7139291a4c  (/lib/libonnxruntime.so.1.18.1+0x25ea4c)
    #6 0x7f7139268e34  (/lib/libonnxruntime.so.1.18.1+0x235e34)
    #7 0x55e7b70402ae in Ort::Env::Env(OrtLoggingLevel, char const*) /usr/include/onnxruntime_cxx_inline.h:481
    #8 0x55e7b703f820 in main /home/ejen/Downloads/onnxruntime_ex/main.cpp:228
    #9 0x7f7138a83082 in __libc_start_main ../csu/libc-start.c:308
    #10 0x55e7b703f62d in _start (/home/ejen/Downloads/onnxruntime_ex/build/Yolov8OnnxRuntimeCPPInference+0x262d)

SUMMARY: AddressSanitizer: 256 byte(s) leaked in 2 allocation(s).

Urgency

Need to figure it out in order to integrate it into the project

Platform

Linux

OS Version

Mint 20.3

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.18.1

ONNX Runtime API

C++

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Model File

No response

Is this a quantized model?

Yes

yuslepukhin commented 3 weeks ago

Ort::Env is a singleton that is only needed once for process lifetime, and you cannot create it more than once. However, attempting to destroy it would cause some DLLs unload. A small piece of memory is leaked intentionally and will not lead to any adverse consequences given it is a singlton.

EjenY-Poltavchiny commented 2 weeks ago

Isn't this strange? I mean intentional memory leakage is not something that is meant to be a "normal thing". Is there some more complicated explanation why things are "that way" or why we can't fix this leakage.

microsoft / onnxruntime