search

rwesson / ALFA

Automated Line Fitting Algorithm
http://www.nebulousresearch.org/codes/alfa/
GNU General Public License v3.0
12 stars 6 forks source link

Output for 2D and 3D FITS files #10

Closed rwesson closed 4 years ago

rwesson commented 8 years ago

ALFA always writes its results out to plain text files. For 2D and 3D FITS files, this can result in thousands or tens of thousands of files in the output directory. It would be much better if this output was written to FITS files.

A file can't be written to by multiple threads simultaneously, so the master thread should write all the results to files, every 100 spectra or something.

The fits themselves can be written out simply to a FITS file with the same dimensionality as the input file. For the line lists, a FITS file with the wavelength axis defined as a lookup table would mean that the result could be an image cube with maps at every wavelength where a line was detected somewhere.

weiaip commented 7 years ago

Are you sure that one cannot simply write to one (or two) FITS output file(s) concurrently, if using proper locking? Some !OMP CRITICAL around the calls should help to not need to make the code unnecessarily complex and get away from the many files. But then, I haven't looked at the code...

(Edited, because before I had read https://heasarc.gsfc.nasa.gov/docs/software/fitsio/c/c_user/node15.html only partially...)

rwesson commented 4 years ago

The situation now is that a single FITS file is written for each pixel analysed. The FITS file contains tables for the fit and the line list. Extracting maps of all lines and all quantities from a directory containing ~10,000 files takes only a few minutes. So I will not attempt to get ALFA to write everything to a single file but will shortly include additional code to facilitate mapping.