Open beelabcloud opened 2 years ago
RyanUnderhill
commented
2 years ago As long as the ORT object destructors are being called (which call the C OrtApi::Release* methods) the memory is released immediately to the OS. There is no garbage collection inside of Onnxruntime.
Can you give more details of what you're looking at? Which OS, and which memory usage metric?
beelabcloud
commented
2 years ago session = new Ort::Session(*env, model_data, (size_t)model_data_length, sessionOptions);
If you do new, then you are responsible to call delete. Ort obejcts are designed to be instantiated by value, so the destructors are automatically called and clean up native resources. They are very much like smart pointers. One usually does not call new on, say std::unique_ptr.
beelabcloud
commented
2 years ago By the way, the comsumption is not caused by the inference, and the program can run flawlessly for a few days, and not eating any extra memory. Thank you very much.
beelabcloud
commented
2 years ago bool MessageLite::ParseFrom(const T& input) {
if (flags & kParse) Clear();
constexpr bool alias = (flags & kMergeWithAliasing) != 0;
return internal::MergeFromImpl
beelabcloud
commented
2 years ago Should I call ::google::protobuf::ShutdownProtobufLibrary() for each cycle?
tianleiwu
commented
2 years ago Is it that Kparse means keep and parse?
I search kParse and kMergeWithAliasing in onnxruntime code, there is no result found. Is the example code from OnnxRuntime?
Should I call ::google::protobuf::ShutdownProtobufLibrary() for each cycle?
I have no experience on this so I cannot answer it. You can have a try. It might introduce other issue.
beelabcloud
commented
2 years ago Hi tianlei, thank you. "kParse" is argument used in the model loading codes in file "cmake\external\protobuf\src\google\protobuf\message_lite.h". ::google::protobuf::ShutdownProtobufLibrary() would shudwon Protobuf entirely as I read. I'm pretty sure that the model data array is retained, and it would be recycled by the OS garbage collection after a long time idle.
ns-wxin
commented
1 year ago I rebuilt onnxruntime with ASAN and got the following leak stack. Wonder if anyone shed some lights on how to release the memory correctly.
Indirect leak of 8 byte(s) in 1 object(s) allocated from:
#1 0x7f228698e5c2 in void std::vector<onnxruntime::NodeArg const*, std::allocator<onnxruntime::NodeArg const*> >::_M_realloc_insert<onnxruntime::NodeArg const* const&>(__gnu_cxx::__normal_iterator<onnxruntime::NodeArg const**, std::vector<onnxruntime::NodeArg const*, std::allocator<onnxruntime::NodeArg const*> > >, onnxruntime::NodeArg const* const&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x25f05c2)
#2 0x7f22869cd4c7 in onnxruntime::Graph::InitializeStateFromModelFileGraphProto() (/opt/3p/lib/libonnxruntime.so.1.12.1+0x262f4c7)
#3 0x7f2286a28311 in onnxruntime::Graph::Graph(onnxruntime::Model const&, onnx::GraphProto*, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > > const&, long, std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection>, onnxruntime::Graph*, onnxruntime::Node const*, onnxruntime::logging::Logger const&, bool) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x268a311)
#4 0x7f2286a29949 in onnxruntime::Graph::Graph(onnxruntime::Model const&, onnx::GraphProto*, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, int> > > const&, long, std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection>, onnxruntime::logging::Logger const&, bool) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x268b949)
#5 0x7f2286ae3bfe in onnxruntime::Model::Model(onnx::ModelProto&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::list<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection>, std::allocator<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection> > > const*, onnxruntime::logging::Logger const&, onnxruntime::ModelOptions const&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x2745bfe)
#6 0x7f2286ae81f7 in onnxruntime::Model::Load(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<onnxruntime::Model>&, std::__cxx11::list<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection>, std::allocator<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection> > > const*, onnxruntime::logging::Logger const&, onnxruntime::ModelOptions const&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x274a1f7)
#7 0x7f2286ae8af4 in onnxruntime::common::Status onnxruntime::LoadModelHelper<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, onnxruntime::LoadModel<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<onnxruntime::Model>&, std::__cxx11::list<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection>, std::allocator<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection> > > const*, onnxruntime::logging::Logger const&, onnxruntime::ModelOptions const&)::{lambda(int)#1}>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, onnxruntime::LoadModel<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<onnxruntime::Model>&, std::__cxx11::list<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection>, std::allocator<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection> > > const*, onnxruntime::logging::Logger const&, onnxruntime::ModelOptions const&)::{lambda(int)#1}) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x274aaf4)
#8 0x7f2286ae97cc in onnxruntime::Model::Load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<onnxruntime::Model>&, std::__cxx11::list<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection>, std::allocator<std::shared_ptr<onnxruntime::IOnnxRuntimeOpSchemaCollection> > > const*, onnxruntime::logging::Logger const&, onnxruntime::ModelOptions const&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x274b7cc)
#9 0x7f2284b2aec2 in std::_Function_handler<onnxruntime::common::Status (std::shared_ptr<onnxruntime::Model>&), onnxruntime::InferenceSession::Load<char>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda(std::shared_ptr<onnxruntime::Model>&)#1}>::_M_invoke(std::_Any_data const&, std::shared_ptr<onnxruntime::Model>&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x78cec2)
#10 0x7f2284b7a3e5 in onnxruntime::InferenceSession::Load(std::function<onnxruntime::common::Status (std::shared_ptr<onnxruntime::Model>&)>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x7dc3e5)
#11 0x7f2284b7b582 in onnxruntime::common::Status onnxruntime::InferenceSession::Load<char>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x7dd582)
#12 0x7f2284ba2002 in onnxruntime::InferenceSession::Load(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x804002)
#13 0x7f2284a051b6 in (anonymous namespace)::CreateSessionAndLoadModel(OrtSessionOptions const*, OrtEnv const*, char const*, void const*, unsigned long, std::unique_ptr<onnxruntime::InferenceSession, std::default_delete<onnxruntime::InferenceSession> >&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x6671b6)
#14 0x7f2284a08364 in OrtApis::CreateSession(OrtEnv const*, char const*, OrtSessionOptions const*, OrtSession**) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x66a364)
#15 0x5f0bdb in Ort::Session::Session(Ort::Env&, char const*, Ort::SessionOptions const&) /opt/3p/include/onnxruntime/core/session/onnxruntime_cxx_inline.h:596
#16 0x5f0bdb in fingerprint::ImageFingerprintGenerator::loadModel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ../libs/dlp/fp20svc/src/ImageFingerprintGenerator.cpp:84
#17 0x5f2945 in fingerprint::ImageFingerprintGenerator::ImageFingerprintGenerator(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) ../libs/dlp/fp20svc/src/ImageFingerprintGenerator.cpp:43
#18 0x45b619 in void std::_Construct<fingerprint::ImageFingerprintGenerator, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(fingerprint::ImageFingerprintGenerator*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/stl_construct.h:119
#19 0x45b619 in void std::allocator_traits<std::allocator<void> >::construct<fingerprint::ImageFingerprintGenerator, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::allocator<void>&, fingerprint::ImageFingerprintGenerator*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/alloc_traits.h:635
#20 0x45b619 in std::_Sp_counted_ptr_inplace<fingerprint::ImageFingerprintGenerator, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:604
#21 0x45b619 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<fingerprint::ImageFingerprintGenerator, std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(fingerprint::ImageFingerprintGenerator*&, std::_Sp_alloc_shared_tag<std::allocator<void> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:971
#22 0x45b619 in std::__shared_ptr<fingerprint::ImageFingerprintGenerator, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::_Sp_alloc_shared_tag<std::allocator<void> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:1712
#23 0x45b619 in std::shared_ptr<fingerprint::ImageFingerprintGenerator>::shared_ptr<std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::_Sp_alloc_shared_tag<std::allocator<void> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:464
#24 0x45b619 in std::shared_ptr<fingerprint::ImageFingerprintGenerator> std::make_shared<fingerprint::ImageFingerprintGenerator, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:1010
#25 0x45b619 in fingerprint::FingerprintGeneratorPool::FingerprintGeneratorPool(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) ../libs/dlp/fp20svc/ImageFingerprintGenerator.hpp:137
#26 0x45b619 in void std::_Construct<fingerprint::FingerprintGeneratorPool, int, char const (&) [46], int>(fingerprint::FingerprintGeneratorPool*, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/stl_construct.h:119
#27 0x45b619 in void std::allocator_traits<std::allocator<void> >::construct<fingerprint::FingerprintGeneratorPool, int, char const (&) [46], int>(std::allocator<void>&, fingerprint::FingerprintGeneratorPool*, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/alloc_traits.h:635
#28 0x45b619 in std::_Sp_counted_ptr_inplace<fingerprint::FingerprintGeneratorPool, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<int, char const (&) [46], int>(std::allocator<void>, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:604
#29 0x45b619 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<fingerprint::FingerprintGeneratorPool, std::allocator<void>, int, char const (&) [46], int>(fingerprint::FingerprintGeneratorPool*&, std::_Sp_alloc_shared_tag<std::allocator<void> >, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:971
#30 0x45b619 in std::__shared_ptr<fingerprint::FingerprintGeneratorPool, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, int, char const (&) [46], int>(std::_Sp_alloc_shared_tag<std::allocator<void> >, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:1712
#31 0x45b619 in std::shared_ptr<fingerprint::FingerprintGeneratorPool>::shared_ptr<std::allocator<void>, int, char const (&) [46], int>(std::_Sp_alloc_shared_tag<std::allocator<void> >, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:464
#32 0x45b619 in std::shared_ptr<fingerprint::FingerprintGeneratorPool> std::make_shared<fingerprint::FingerprintGeneratorPool, int, char const (&) [46], int>(int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:1010
#33 0x445036 in __static_initialization_and_destruction_0 ../libs/dlp/fp20svc/test/src/DlpFp20Test.cpp:35
#34 0x6ccafc in __libc_csu_init (/home/wxin/src/ns/dataplane/obj-asan/dlp_fp20_lib_test+0x6ccafc)@ns-wxin Could you use a more recent build? It looks like you're using onnxruntime 1.12.1? Is the leak only 8 bytes?
ns-wxin
commented
1 year ago Yeah. It's 1.12.1. There's various leaks detected such as the following leaking 536 bytes. Looks like the default constructor of SessionOptions allocated some space but not released. I tried Ort::SessionOptions.release() in my destructor but it didn't seem to get rid of this leak.
Direct leak of 536 byte(s) in 1 object(s) allocated from:
#1 0x7f26711d1287 in OrtApis::CreateSessionOptions(OrtSessionOptions**) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x6ff287)
#2 0x5f24b2 in Ort::SessionOptions::SessionOptions() /opt/3p/include/onnxruntime/core/session/onnxruntime_cxx_inline.h:412
#3 0x5f24b2 in fingerprint::ImageFingerprintGenerator::ImageFingerprintGenerator(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) ../libs/dlp/fp20svc/src/ImageFingerprintGenerator.cpp:40
#4 0x45b619 in void std::_Construct<fingerprint::ImageFingerprintGenerator, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(fingerprint::ImageFingerprintGenerator*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/stl_construct.h:119
#5 0x45b619 in void std::allocator_traits<std::allocator<void> >::construct<fingerprint::ImageFingerprintGenerator, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::allocator<void>&, fingerprint::ImageFingerprintGenerator*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/alloc_traits.h:635
#6 0x45b619 in std::_Sp_counted_ptr_inplace<fingerprint::ImageFingerprintGenerator, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:604
#7 0x45b619 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<fingerprint::ImageFingerprintGenerator, std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(fingerprint::ImageFingerprintGenerator*&, std::_Sp_alloc_shared_tag<std::allocator<void> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:971
#8 0x45b619 in std::__shared_ptr<fingerprint::ImageFingerprintGenerator, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::_Sp_alloc_shared_tag<std::allocator<void> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:1712
#9 0x45b619 in std::shared_ptr<fingerprint::ImageFingerprintGenerator>::shared_ptr<std::allocator<void>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::_Sp_alloc_shared_tag<std::allocator<void> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:464
#10 0x45b619 in std::shared_ptr<fingerprint::ImageFingerprintGenerator> std::make_shared<fingerprint::ImageFingerprintGenerator, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:1010
#11 0x45b619 in fingerprint::FingerprintGeneratorPool::FingerprintGeneratorPool(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int) ../libs/dlp/fp20svc/ImageFingerprintGenerator.hpp:137
#12 0x45b619 in void std::_Construct<fingerprint::FingerprintGeneratorPool, int, char const (&) [46], int>(fingerprint::FingerprintGeneratorPool*, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/stl_construct.h:119
#13 0x45b619 in void std::allocator_traits<std::allocator<void> >::construct<fingerprint::FingerprintGeneratorPool, int, char const (&) [46], int>(std::allocator<void>&, fingerprint::FingerprintGeneratorPool*, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/alloc_traits.h:635
#14 0x45b619 in std::_Sp_counted_ptr_inplace<fingerprint::FingerprintGeneratorPool, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<int, char const (&) [46], int>(std::allocator<void>, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:604
#15 0x45b619 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<fingerprint::FingerprintGeneratorPool, std::allocator<void>, int, char const (&) [46], int>(fingerprint::FingerprintGeneratorPool*&, std::_Sp_alloc_shared_tag<std::allocator<void> >, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:971
#16 0x45b619 in std::__shared_ptr<fingerprint::FingerprintGeneratorPool, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, int, char const (&) [46], int>(std::_Sp_alloc_shared_tag<std::allocator<void> >, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:1712
#17 0x45b619 in std::shared_ptr<fingerprint::FingerprintGeneratorPool>::shared_ptr<std::allocator<void>, int, char const (&) [46], int>(std::_Sp_alloc_shared_tag<std::allocator<void> >, int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:464
#18 0x45b619 in std::shared_ptr<fingerprint::FingerprintGeneratorPool> std::make_shared<fingerprint::FingerprintGeneratorPool, int, char const (&) [46], int>(int&&, char const (&) [46], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:1010
#19 0x445036 in __static_initialization_and_destruction_0 ../libs/dlp/fp20svc/test/src/DlpFp20Test.cpp:35
#20 0x6ccafc in __libc_csu_init (/home/wxin/src/ns/dataplane/obj-asan/dlp_fp20_lib_test+0x6ccafc)Sorry, the un-necessary release() on session caused most of the leak. Now that is fixed, I'm only seeing the leaking when using extensions. Specifically on CustomDomain related code. The code looks like the following:
`TextMLFingerprintGenerator::TextMLFingerprintGenerator(std::string modelFile, int threadNum) : m_modelFile(std::move(modelFile)), m_modelThreads(threadNum) { m_modelSession = loadModel(m_modelFile); m_memoryInfo = Ort::MemoryInfo::CreateCpu(OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);
// setting input name for the model
m_inputName = TEXT_ML_MODEL_INPUT_NAME.c_str();
// setting input dimensions
m_inputDims = {1};
// setting output names
m_outputName = TEXT_ML_MODEL_OUTPUT_NAME.c_str();
m_outputDims = {1, TEXT_ML_EMBEDDING_LENGTH};}
std::shared_ptr
m_sessionOptions.SetIntraOpNumThreads(m_modelThreads);
m_sessionOptions.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_ALL);
m_sessionOptions.DisableCpuMemArena();
m_sessionOptions.EnableOrtCustomOps();
std::shared_ptr<Ort::Session> session =
std::make_shared<Ort::Session>(m_env, modelFile.c_str(), m_sessionOptions);
LOG(INFO) << "Loaded model: " << modelFile;
return session;}`
==1926430==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 56 byte(s) in 1 object(s) allocated from:
#1 0x7fbfcccd7d42 in OrtApis::CreateCustomOpDomain(char const*, OrtCustomOpDomain**) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x624d42)
#2 0x7fbfcf3cbb16 in RegisterCustomOps (/opt/3p/lib/libonnxruntime.so.1.12.1+0x2d18b16)
#3 0x7fbfccce24f0 in OrtApis::EnableOrtCustomOps(OrtSessionOptions*) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x62f4f0)
#4 0x605318 in Ort::SessionOptions::EnableOrtCustomOps() /opt/3p/include/onnxruntime/core/session/onnxruntime_cxx_inline.h:452
#5 0x605318 in fingerprint::TextMLFingerprintGenerator::loadModel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ../libs/dlp/fp20svc/src/TextMLFingerprintGenerator.cpp:88
#6 0x606d9a in fingerprint::TextMLFingerprintGenerator::TextMLFingerprintGenerator(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) ../libs/dlp/fp20svc/src/TextMLFingerprintGenerator.cpp:64
#7 0x458605 in void std::_Construct<fingerprint::TextMLFingerprintGenerator, char const (&) [54], int>(fingerprint::TextMLFingerprintGenerator*, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/stl_construct.h:119
#8 0x458605 in void std::allocator_traits<std::allocator<void> >::construct<fingerprint::TextMLFingerprintGenerator, char const (&) [54], int>(std::allocator<void>&, fingerprint::TextMLFingerprintGenerator*, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/alloc_traits.h:635
#9 0x458605 in std::_Sp_counted_ptr_inplace<fingerprint::TextMLFingerprintGenerator, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<char const (&) [54], int>(std::allocator<void>, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:604
#10 0x458605 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<fingerprint::TextMLFingerprintGenerator, std::allocator<void>, char const (&) [54], int>(fingerprint::TextMLFingerprintGenerator*&, std::_Sp_alloc_shared_tag<std::allocator<void> >, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:971
#11 0x458605 in std::__shared_ptr<fingerprint::TextMLFingerprintGenerator, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, char const (&) [54], int>(std::_Sp_alloc_shared_tag<std::allocator<void> >, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:1712
#12 0x458605 in std::shared_ptr<fingerprint::TextMLFingerprintGenerator>::shared_ptr<std::allocator<void>, char const (&) [54], int>(std::_Sp_alloc_shared_tag<std::allocator<void> >, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:464
#13 0x458605 in std::shared_ptr<fingerprint::TextMLFingerprintGenerator> std::make_shared<fingerprint::TextMLFingerprintGenerator, char const (&) [54], int>(char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:1010
#14 0x44506d in __static_initialization_and_destruction_0 ../libs/dlp/fp20svc/test/src/DlpFp20Test.cpp:39
#15 0x6cc3dc in __libc_csu_init (/home/wxin/src/ns/dataplane/obj-asan/dlp_fp20_lib_test+0x6cc3dc)and
Indirect leak of 256 byte(s) in 1 object(s) allocated from:
#1 0x7fbfcccfa1b2 in void std::vector<OrtCustomOp const*, std::allocator<OrtCustomOp const*> >::_M_realloc_insert<OrtCustomOp const*&>(__gnu_cxx::__normal_iterator<OrtCustomOp const**, std::vector<OrtCustomOp const*, std::allocator<OrtCustomOp const*> > >, OrtCustomOp const*&) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x6471b2)
#2 0x7fbfcccfa72c in OrtApis::CustomOpDomain_Add(OrtCustomOpDomain*, OrtCustomOp const*) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x64772c)
#3 0x7fbfcf3cc48f in RegisterCustomOps (/opt/3p/lib/libonnxruntime.so.1.12.1+0x2d1948f)
#4 0x7fbfccce24f0 in OrtApis::EnableOrtCustomOps(OrtSessionOptions*) (/opt/3p/lib/libonnxruntime.so.1.12.1+0x62f4f0)
#5 0x605318 in Ort::SessionOptions::EnableOrtCustomOps() /opt/3p/include/onnxruntime/core/session/onnxruntime_cxx_inline.h:452
#6 0x605318 in fingerprint::TextMLFingerprintGenerator::loadModel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ../libs/dlp/fp20svc/src/TextMLFingerprintGenerator.cpp:88
#7 0x606d9a in fingerprint::TextMLFingerprintGenerator::TextMLFingerprintGenerator(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) ../libs/dlp/fp20svc/src/TextMLFingerprintGenerator.cpp:64
#8 0x458605 in void std::_Construct<fingerprint::TextMLFingerprintGenerator, char const (&) [54], int>(fingerprint::TextMLFingerprintGenerator*, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/stl_construct.h:119
#9 0x458605 in void std::allocator_traits<std::allocator<void> >::construct<fingerprint::TextMLFingerprintGenerator, char const (&) [54], int>(std::allocator<void>&, fingerprint::TextMLFingerprintGenerator*, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/alloc_traits.h:635
#10 0x458605 in std::_Sp_counted_ptr_inplace<fingerprint::TextMLFingerprintGenerator, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<char const (&) [54], int>(std::allocator<void>, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:604
#11 0x458605 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<fingerprint::TextMLFingerprintGenerator, std::allocator<void>, char const (&) [54], int>(fingerprint::TextMLFingerprintGenerator*&, std::_Sp_alloc_shared_tag<std::allocator<void> >, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:971
#12 0x458605 in std::__shared_ptr<fingerprint::TextMLFingerprintGenerator, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, char const (&) [54], int>(std::_Sp_alloc_shared_tag<std::allocator<void> >, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr_base.h:1712
#13 0x458605 in std::shared_ptr<fingerprint::TextMLFingerprintGenerator>::shared_ptr<std::allocator<void>, char const (&) [54], int>(std::_Sp_alloc_shared_tag<std::allocator<void> >, char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:464
#14 0x458605 in std::shared_ptr<fingerprint::TextMLFingerprintGenerator> std::make_shared<fingerprint::TextMLFingerprintGenerator, char const (&) [54], int>(char const (&) [54], int&&) /opt/gcc-12/include/c++/12.2.0/bits/shared_ptr.h:1010
#15 0x44506d in __static_initialization_and_destruction_0 ../libs/dlp/fp20svc/test/src/DlpFp20Test.cpp:39
#16 0x6cc3dc in __libc_csu_init (/home/wxin/src/ns/dataplane/obj-asan/dlp_fp20_lib_test+0x6cc3dc)
I 'm using ORT C++ API to infer several models sequentialy. The consumption of memory is pretty much, so I try to unload some sessones after inference. However the consumption is not fully freed, and no memory leak detected. And I notice that the memory will be freed by the OS after a while (1 hour background alive), I guess, it might be recycled by the mechanism of system garbage collection. So I am wondering it is there some magic API to recycle the garbage, and I would like to know what cause the memory residents, the threads pooling or the weights? Is there a way to tell the OS to do recycle instantly? Thanks.