PaulHancock / Aegean

The Aegean source finding program and associated tools
http://aegeantools.rtfd.io/
Other
47 stars 14 forks source link

Image size limit? #3

Closed PaulHancock closed 8 years ago

PaulHancock commented 10 years ago

When Aegean is run on a large image it crashes and also brings down the term.app that it is running in.

Mosaic-1.fits size: 1.296322560 GB 18000*18000 pixels

Computer Spec: Processor - 2.9 GHz Intel Core i7 Memory - 8 GB

Command line input: python aegean.py Mosaic-1.fits --rms=35e-6 --outfile=AegOut.M1 --table=AegM1.reg

(Using Aegean.r982)

PaulHancock commented 10 years ago

Aegean doesn't have any explicit limits on image size however the are a few things that could be causing this problem. I suspect that the cause of the issue is that Aegean is simply using too much ram.

Aegean ends up with 4 arrays that are all the same size as the input image. One for the noise, one for the background, one for the curvature, and then the image itself. Therefore it will use up to 4x the image size in ram.

There is also an issue with the fits module from astropy whereby it imports images as float64 even when the image header specifies a bitpix of 32. This will effectively double the size of all the images.

The first of these issues will need some work to change the order in which Aegean loads/analyses images and fits islands. This will take some time.

The second issue should require only a small amount of re-factoring at the point where the images are loaded.