search

Caltech-IPAC / Montage

Image Mosaics for Astronomers
Other
113 stars 48 forks source link

Memory leak in mDiff #68

Open guillaumechereau opened 1 year ago

guillaumechereau commented 1 year ago

It seems that the mDiff function leaks memory in the case where "All pixels are blank" (line 949 of file montageDiff.c).

This is an issue for me because I try to run mDiffExec on a large number of files and the command eventually runs out of memory.

Regards, G

pandapawskat commented 8 months ago

I've found that a lot of these tools have memory leaks or other memory management issues that aren't a problem when run as an individual tool, but add up quickly when using the python wrappers. Calling mProjectQL in a loop eats up memory, and mViewer starts having issues like double frees if called repeatedly. My solution has been to use the multiprocessing library to fire off the methods in a separate process so everything gets cleaned up when it terminates. Definitely not ideal but it got me unstuck.

pandapawskat commented 8 months ago

Just adding that mAdd seems to leave stuff in globals, so later runs are affected by the previous ones. I think I'm going to refactor all the montagePy calls into separate processes - who knows what other bugs I may have due to things like this.