Increased memory consumption when loading OpenCV library (8GBs in v1.5.4 compared to 2.2GBs in v1.5.3)

[lukaszbachman]

Hello team!

I've recently noticed what seems to be a much larger memory footprint of one of the features that we have in our project which uses JavaCV+OpenCV for image analysis. We recently upgraded from 1.5.3 to 1.5.7 and the issue went unnoticed for a while, but during more extensive tests we started noticing that it brings our machines to their knees.

The problem manifests itself as soon as we execute the code which imports org.bytedeco.opencv.opencv_core from OpenCV, which in turn loads the libraries via:

static {
         Loader.load();
}

The underlying process then immediately spikes to 8GBs of Virtual RAM Memory. Since our application can launch up to 8 of such processes at a time, it's a killer for our machines.

I tried running the exact same code but instead of loading OpenCV I just loaded one of classes from FFMpeg - the memory footprint stayed around 700MBs mark. I also checked version 1.5.3 and then loading of OpenCV claimed 2.2GBs of memory - still a lot, but more manageable.

Here are a few screenshots of what I'm seeing in processes for different versions (highlighted process is my test):

JavaCV v1.5.3

JavaCV v1.5.7

JavaCV v1.5.8

For the test I used javacv-platform artifact, although in production we are using trimmed down version of dependencies. I ran all tests with JDK17 with -Xms50m -Xmx100m VM settings and debugger enabled (sample output from 1.5.8 attached).

My testing project where this could be easily reproduced: https://github.com/lukaszbachman/javacv-memtest/blob/main/src/main/java/io/bitmasters/MemTest.java

JavaCV log from an attempt of running it with v1.5.8: https://github.com/lukaszbachman/javacv-memtest/blob/main/javacv-1.5.8.log

Let me know if you need any more input to comments on this. Big thanks!

[saudet]

Could you narrow down from which release this started happening?

[lukaszbachman]

Could you narrow down from which release this started happening?

I'm on it! Thanks for a quick reply!

[lukaszbachman]

1.5.4 is ok

1.5.5 is NOT ok

So it looks like it started happening with 1.5.5. I've uploaded debug logs from 1.5.4 vs 1.5.5 into my test project.

• 1.5.4 logs
• 1.5.5 logs

[saudet]

Hum, the biggest change between 1.5.5 and 1.5.4 for OpenCV on Windows is the move from Visual Studio 2017 to 2019: https://github.com/bytedeco/javacpp-presets/releases/tag/1.5.5 There hasn't been a lot of changes w.r.t to OpenCV in 1.5.5, so I don't see what else it could be... But if that's the case, that's something we'd need to report upstream: https://developercommunity.visualstudio.com/home

[lukaszbachman]

So since you mentioned Windows I thought I may run the same test on my Ubuntu machine. Results are a bit different, but also not great.

With 1.5.4 I can already see large memory footprint in Virtual Memory:

With 1.5.5 it gets a bit bigger again:

I'm not really sure what to make out of it. I know it's "only" virtual memory, but still it may contribute to a reduction in overall system performance.

@saudet do you think I could do any more tests or provide more debug information? Or perhaps there is a way to only load part of the OpenCV library?

[saudet]

As usual, please try to ask upstream: https://github.com/opencv/opencv

[lukaszbachman]

Ok, thanks for the hint and looking into it.

At the moment the easier way around it was to use Java API from OpenCV directly. We only had one class to be changed to make it work and it seems that it doesn't have similar memory overhead as compared to using JavaCV as a wrapper. Obviously that may not be a way forward for other developers, but it was a sensible approach for us.

I'll close the ticket for now.

Thanks for the support!

[saudet]

If you're not seeing the same behavior with a build without OpenBLAS, it's possible that the problem comes from OpenBLAS. Please try to load only OpenBLAS see what that gives you.

@HGuillemet Have you ever noticed that about OpenBLAS?

[HGuillemet]

Nothing that gave me trouble. Some remarks:

• Virtual memory can increase just because more libraries are loaded, thoses libraries bringing their own libraries against which they are linked to the virtual space. If they stay unused they shouldn't be loaded in physical memory, and cost nothing significative in terms of resources.

• Such big difference in windows could well be related to BLAS preset indeed, loading either MKL or another lighter library. Or maybe related to the way MKL is loaded.

• On Linux it's easy to see what fills the virtual space using /proc/<process id>/maps. I guess something similar must exist for Windows. Maybe this : VMMap. Can you try it on your process and see what changes when it jumps from 2G to 8G ?

[saudet]

@HGuillemet I'm not seeing anything in /proc/xxx/maps, but I've confirmed that the allocation is caused by OpenBLAS. With MKL, it's a bit better at ~8GB instead of ~9GB, but still pretty large.

@lukaszbachman Unfortunately, BLAS and LAPACK are a crucial components to get performance out of OpenCV, so we're not going to disable them in the default builds, that is unless you are able to offer proof that they adversely affect performance, which I doubt very much is the case given how widely used BLAS and LAPACK are in the industry.

[saudet]

Duplicate of #1329.