Memory leak when preprocessing the lidar data

[Ianpengg]

Hello I want to report a problem when pre-processing the data with the utils/auto_gen_residual_images.py I've noticed that there seems to be a memory leak issue when running this code, as memory usage continues to increase until the code crashes. I've found it was caused by the stage that saving the visualize images.

---

By default, the visualize parameter is set to True, which automatically saves visualization images. However, when using the plt.close() function to clear the current figure, the function did not seem to function properly, potentially contributing to the memory leak issue.

# utils/auto_gen_residual_images.py 
...
if visualize:
    fig = plt.figure(frameon=False, figsize=(16, 10))
    fig.set_size_inches(20.48, 0.64)
    ax = plt.Axes(fig, [0., 0., 1., 1.])
    ax.set_axis_off()
    fig.add_axes(ax)
    ax.imshow(diff_image, vmin=0, vmax=1)
    image_name = os.path.join(visualization_folder, str(frame_idx).zfill(6))
    plt.savefig(image_name)
    plt.close()

I solved this issue by set the visualize to False . But I think this problem should be addressed , thanks

System information:

Operating system: Ubuntu 20.04 Python version: 3.7.5 Libraries used: matplotlib=3.5.3, os

I would appreciate any advice or insights on how to resolve this issue.

[MaxChanger]

Hello, I didn't notice this problem before, I want to confirm, how much memory does your PC have?

In addition, this figure is only used for visualization, and the program actually loads the np.save() files, so setting it to False does not affect subsequent development.

And, I think this matplotlib/issues/20490 might be helpful.

[Ianpengg]

Hello thanks for replying.

I have a PC with 64GB RAM, and 24GB was already in use before I ran the code. When I started processing the first sequence, the memory usage kept increasing until it reached about 63.6GB, at which point the program stopped due to running out of memory.

To address the memory usage issue, I decided not to save the visualization figures, and now the memory usage is stable and remains the same as when I first ran the code.

[MaxChanger]

Hello, I think this problem is more inclined to find a solution in matplotlib, I will temporarily close, if there is still a problem or there is a new solution welcome to open again.