basler / pypylon

The official python wrapper for the pylon Camera Software Suite
http://www.baslerweb.com
BSD 3-Clause "New" or "Revised" License
558 stars 207 forks source link

Memory Accumulation #313

Open zamarjavier opened 3 years ago

zamarjavier commented 3 years ago

Hello,

I have two Usb camera basler and I'm trying to obtain a synchronization capture using a software trigger. After that, I process the images in each camera. The idea is to execute multiple launches doing this work for each launch. I achieve this goal but now, I have another problem. I have a memory accumulation. For example after running the code my RAM accumulates 3GB. Then after the first launch, the RAM accumulates 5,6. After the second launch the RAM accumulates 8,2 and so on.

I'm using threads in python. 2 for each camera, one for capturing and the other for processing. I don't know how to solve this problem, does anybody know how to do it?

Thanks in advance.

Kind regards

thiesmoeller commented 3 years ago

What OS do you run?

For linux I would recommend to use for further analysis: "GitHub - KDE/heaptrack: A heap memory profiler for Linux" https://github.com/KDE/heaptrack

And swich the python allocator to the c library by setting

PYTHONMALLOC=malloc

In the environment

zamarjavier commented 3 years ago

Thanks for your response @thiesmoeller

I'm using Ubuntu 18 LTS.

I think that I have detected the problem but I don't know how to solve it. I'm capturing 600 images for each camera and the memory grows when I extract the image form grabresult and insert into a list. Here you can see an example:

a = [] while camera.IsGrabbing(): grab_result = camera.RetrieveResult(10000, pylon.TimeoutHandling_ThrowException) if grab_result.GrabSucceeded(): a.append(grab_result.GetArray()

I need the images into a list, because when de capture process ends, I need to process them.

Do you know how to do it whithout memory accumulation.

On the other hand, how can I do what you mention in your message: And swich the python allocator to the c library by setting

PYTHONMALLOC=malloc

In the environment

Thanks for your response.

thiesmoeller commented 3 years ago

if the images are not garbage collected, it seems, that some code of you still holds a reference to the list 'a' Do you really have to hold 600 images in memory? or would it be possible to process them while grabbing?

For the grabbing, you don't have to use your own thread. pypylon already has an efficient c++ implementiation of a background grab loop -> see https://github.com/basler/pypylon/blob/master/samples/grabusinggrabloopthread.py

( the PYTHONMALLOC was only mentioned if you want to start to really debug your system, because the python3 pool memory allocator normally irritates memory debugging tools, and since 3.8 it is possible to disable it while debugging )