[Bug]: Memory not free well

[xiaoweiChen]

OpenVINO Version

2024.4.0-16579-c3152d32c9c-releases/2024/4

Operating System

Windows System

Device used for inference

AUTO

Framework

None

Model used

custom

Issue description

When the program finish at return 0;, the memory is not free well, I think.

CPU: 11th Gen Intel(R) Core(TM) i5-1145G7 @ 2.60GHz

GPU 0: Intel(R) Iris(R) Xe Graphics driver version: 31.0.101.5388

Step-by-step reproduction

The model in this package model.zip

Code :

#include "openvino/openvino.hpp"

#include <stdexcept>
#include <iostream>
#include <string>

const std::string test_model_path = "ladon_360p_input_re.xml";
//const std::string device = "CPU";
const std::string device = "AUTO";

//ov::Core ie;
void createNetwork() {
  ov::Core ie;

  auto network = ie.read_model(test_model_path);

  auto executable_network =
    ie.compile_model(
      network
    , device
    );

  auto infer_request = 
    executable_network.create_infer_request();
}

int main() {

  try {
      std::cout << ov::get_openvino_version().buildNumber << std::endl;

      for (int time = 0; time < 1; time++) {
          createNetwork();
      }
  }
  catch (const std::exception& ex) {
    std::cerr << ex.what() << std::endl;
  }
  return 0;
}

CPU or AUTO as device, when I put a breakpoint at return 0; in main function. The Task Manager would show the memory is 27.5MB still using. But, I think would not too many memory need at this time

When I comment out the createNetwork(); line. At same breakpoint position, the memory is 0.6MB still using. I think this is right and what I want.

Relevant log output

No response

Issue submission checklist

• [X] I'm reporting an issue. It's not a question.
• [X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• [X] There is reproducer code and related data files such as images, videos, models, etc.

[praasz]

@xiaoweiChen Thanks for reporting the issue.

The reported memory used at and your application end it doesn't OpenVINO not correctly de-allocated it. It shows how much process is using memory but still inside this value there could be some unused memory but just not collected by OS.

I've done similar test but using Release version of OpenVINO the values were different in Windows system monitor:

• Start: 0.7 MiB; Peak: 62.4 MiB; End: 7.5 MiB; (model from this issue)
• Start: 0.7 MiB; Peak: ; 9380 MiB End: 21.8 MiB; (other big model)

When createNetwork(); is removed from the code nothing from OpenVINO is used and this is reason of low memory usage. The usage at End of test application are because some of thinks used in OpenVINO and others which OpenVINO and user application has limited control and depends from OS.

OpenVINO dependent:

• OpenVINO libraries create some data in lazy way which will live until process end (they give some base memory usage)
• allocate/deallocate memory on heap. I've done similar code check for memory leaks and heap allocation and there is no issues. Some data will live until application end. In my test environment, I've measured about 1.2 MiB.

OpenVINO not dependent:

• Many allocation of small and big chunks of memories. The big chunks are reclaimed by OS correctly but release memory after small chunks can still be left in process (I think it can give most additional usage and end of application). This not used memory can be not reclaimed because of fragmentation and OS logic to keep them to avoid system calls from process for memory allocation as they are costly.
• dynamically load/unload the plugins libraries but OS will decide when remove them completely.

If you don't see any memory leaks (de/allocations by OpenVINO) is not bug but how OS works. To solve it maybe is possible to find some Windows API and tune Windows memory manger to reclaim memory in different way but it can decrease the performance.