search

GeminiDRSoftware / FitsStorage

Gemini FitsStorage codebase. Version 2 following some significant refactoring early 2023
Other
0 stars 0 forks source link

Ghost _calibrated files -fits! warning #45

Open phirstgemini opened 4 months ago

phirstgemini commented 4 months ago

fitsverify complains that there shouldn't be a CRPIX2 header (based on WCSAXES being 1 I think), and astropy wcslint helpfully explodes with a MemoryError. Now I'm not 100% confident that fitsverify is in the right here. It's a bit unusual I think to have WCSAXES < NAXIS, and I don't see anything in the FITS papers that discuss (or forbid) that possibility, so it's all a bit vague. But wcslint blowing up on them perhaps suggests something is not quite standard...

From Chris on slack:

This is not standard, you are correct. The data are 30-something orders (one per row in the data) where each order has its own tabular wavelength solution. There is no FITS standard for this (at least, not that I could find, if you can correct me on that, I would happily adhere to it), so I made one up where the CD matrix points to a location within a LUT of wavelength values (which, in this case, is the same size as the data, with each pixel representing the wavelength of the corresponding pixel in the data). This adheres as much as possible to the -TAB convention in the use of the CD matrix pointing to the location within a table and not saying anything about the actual world coordinate(s). The only FITS compliant way to store these data is to write them as 30-something separate spectra, each with an associated AWAV table. With masks and variance, that's over 130 FITS extensions, each of which has a header larger than the data. I chose not to do this because it seemed inefficient and it would require a massive rewrite of the code.

OK, so basically it's correct behavior for these to be identified as having fits issues. Would be nice to figure out a sensible compliant way to store the WCS. Storing this here for reference.

phirstgemini commented 2 months ago

So this is not a FitsStorage bug, and highlighting the FITS standard violation is correct behavior, ugly though it is in the search results.

As such, no action due. It would be nice to tell DRAGONS how to do this nicely.

Keeping this open as a placeholder and I'm sure it will get queried in the future...