search

neuromorphicsystems / astrometry

Astrometry turns a list of star positions into a pixel-to-sky transformation (WCS)
GNU General Public License v3.0
27 stars 3 forks source link

Memory leak #6

Open juanep97 opened 1 year ago

juanep97 commented 1 year ago

Repeated calls to the astrometry solver (hundreds) end up increasing the allocated memory until it overflow (currently circumvented it in my script by spawning a forked process for each field; which frees the allocated memory).

Versions:

Steps to reproduce:

(thanks for the nice module, btw)

alexw-im commented 12 months ago

Just wanted to say thank you for catching this and mentioning it!

juanep97 commented 11 months ago

I will add a test scenario, if you think it's useful; although I don't know if this will be a problem of astrometry or astrometry.net.

alexw-im commented 11 months ago

I've just noticed that it seems @rlancaste ran across maybe the same memory leaks when adapting Astrometry.net for StellarSolver: https://github.com/dstndstn/astrometry.net/issues/183#issuecomment-638893225

Perhaps @aMarcireau could compare notes to find and fix them if and when it's a priority!

rlancaste commented 11 months ago

Yes, there definitely were some memory leaks. I had to correct a bunch of them as best I could so that KStars did not have significant memory leaks when plate solving. I cannot guarantee that every change I made was the best solution, but I tried my best. I think I managed to fix all the biggest ones. There also were some issues I never resolved, but it got to the point where they were pretty small issues and did not significantly affect performance.

alexw-im commented 11 months ago

Yes, there definitely were some memory leaks. I had to correct a bunch of them as best I could so that KStars did not have significant memory leaks when plate solving.

Do you happen to know whether any of them got up-streamed into the original astrometry.net repo?

Aside - thank you for your work! I've only recently taken up any kind of active interest in astrophotography and the first thing I realized is that 99% of the barrier to entry is the effort in realizing which tools do what you're trying to accomplish and figuring out how to jam them all together. Stellar solver in Kstars really goes a long way on that front.

rlancaste commented 11 months ago

So I know at least one person was mentioning that they wanted to try to upstream some of my changes. I do not have the time to do that now. I am a full time Physics and Computer Science teacher. I had to the time to do StellarSolver during the pandemic, but now that we are back full time and I have a couple of other projects I have been working on, I haven't had time for as much open source coding.

juanep97 commented 9 months ago

Sorry, I'm quite busy lately with academic life... I add a note here for when I have time to dive and check the origin of the memory leak. I realized (2) below also happens:

  1. Memory leaks if the solver is invoked repeatedly many times, even if running for a short time. Memory is not freed.
  2. Memory increases to ~tens of GB only in one run, if position hint separation is high (180 deg).

It seems to be related to the loading of index files, maybe memory is not released after loading them