search

PaulHancock / Aegean

The Aegean source finding program and associated tools
http://aegeantools.rtfd.io/
Other
47 stars 14 forks source link

There is no output running aegean #135

Closed Andywang201605 closed 3 years ago

Andywang201605 commented 4 years ago

I am working with Aegeantools (v2.2.3). When I was trying to run aegean on test file, aegean 1904-66_SIN.fits --cores 10 --table out.csv --debug, it seemed that the processing "stopped" at this stage. (see figure below) I waited for about 20 mins and there was still no any other output.

image

I also ran it for my own fits file, it turned out the same result. Last year I met the same problem, but after setting OMP_NUM_THREADS 1, everything went smoothly. However, it doesn't work this time

PaulHancock commented 4 years ago

Hi Andy,

Thanks for the screenshot. It looks to me like Aegean has found a very large island (29k pixels, 8k masked) with 506 sources. This means that in the background the fitting will be dealing with matricies of size 29k x 29k, which is probably going to be super slow. Since you are using default values for fitting (internal background calculation, 5/4 seed/clip) I don't know what has caused this problem.

From the output you list (which has --debug) it looks like the issue still happens when you use just one core. The fact that OMP_NUM_THREADS 1 avoids the issue is very strange since i'm not explicitly using open mp (though numpy/scipy/multiprocessing might be).

I'll keep an eye on this problem, but I suspect that this will be one of the issues that I can't immediately fix and will (hopefully) go away when I release v3.0 of aegean which will use a different parallelism framework.

Paul.

Andywang201605 commented 4 years ago

Hi Paul,

Thanks! I ran BANE first and ran Aegean with --autoload options, and all is good. I am not sure if there are some issues with aegean - since I used to use aegean directly when finding sources and there was no such error. Hope this information will help you :)

Andy