Support for 64-bit integer FITS files

[drtobybrown]

CARTA is having trouble finding and loading a 3D cube in fits format that python (and DS9) handles fine. The error message is less than helpful.

$ ls testfile.fits 
testfile.fits
$ carta testfile.fits 

Happy to provide the fits file privately on request.

[drtobybrown]

Followed up via email with the file to kswang@asiaa.sinica.edu.tw

[kswang1029]

unhandled exception: FITSImgParser::setup - /Users/kswang/Downloads/testfile.fits Error reading initial record -- exiting. 2020-07-29 01:24:35 WARN FITS card 2: BITPIX = 64 / array data type
2020-07-29 01:24:35 WARN Illegal value for keyword BITPIX. 2020-07-29 01:24:35 WARN Missing required BITPIX keyword 2020-07-29 01:24:35 WARN File /Users/kswang/Downloads/testfile.fits Physical record 1 logical record 1 -- 2020-07-29 01:24:35 WARN + [FitsInput::init()] Unrecognizable record at the beginning Cannot display testfile.fits-raster as raster image: FITSImgParser::setup - /Users/kswang/Downloads/testfile.fits Error reading initial record -- exiting.

Could you try to make the following change?

BITPIX = -32 /Floating point (32 bit)

Currently CARTA does not like BITPIX=64...

[veggiesaurus]

unhandled exception: FITSImgParser::setup - /Users/kswang/Downloads/testfile.fits Error reading initial record -- exiting. 2020-07-29 01:24:35 WARN FITS card 2: BITPIX = 64 / array data type
2020-07-29 01:24:35 WARN Illegal value for keyword BITPIX. 2020-07-29 01:24:35 WARN Missing required BITPIX keyword 2020-07-29 01:24:35 WARN File /Users/kswang/Downloads/testfile.fits Physical record 1 logical record 1 -- 2020-07-29 01:24:35 WARN + [FitsInput::init()] Unrecognizable record at the beginning Cannot display testfile.fits-raster as raster image: FITSImgParser::setup - /Users/kswang/Downloads/testfile.fits Error reading initial record -- exiting.

Could you try to make the following change?

BITPIX = -32 /Floating point (32 bit)

Currently CARTA does not like BITPIX=64...

This looks like it's just the first hdu that is a table hdu. What does the fitsheader util say?

[kswang1029]

There is only one HDU in this fits file.

KARMA-kvis cannot load it either.

Unknown unit: "deg" for: "CUNIT1", dropping Unknown unit: "deg" for: "CUNIT2", dropping Non-standard "CUNIT3" has no effect: dropping Illegal value for BITPIX: 64 foreign_guess_and_read: error reading FITS file Filepopup::filesel_cbk: error reading file: "Downloads/testfile.fits"

JS9 cannot load it.

[veggiesaurus]

unhandled exception: FITSImgParser::setup - /Users/kswang/Downloads/testfile.fits Error reading initial record -- exiting. 2020-07-29 01:24:35 WARN FITS card 2: BITPIX = 64 / array data type 2020-07-29 01:24:35 WARN Illegal value for keyword BITPIX. 2020-07-29 01:24:35 WARN Missing required BITPIX keyword 2020-07-29 01:24:35 WARN File /Users/kswang/Downloads/testfile.fits Physical record 1 logical record 1 -- 2020-07-29 01:24:35 WARN + [FitsInput::init()] Unrecognizable record at the beginning Cannot display testfile.fits-raster as raster image: FITSImgParser::setup - /Users/kswang/Downloads/testfile.fits Error reading initial record -- exiting.

Could you try to make the following change?

BITPIX = -32 /Floating point (32 bit)

Currently CARTA does not like BITPIX=64...

Ah ok, I see. We've tried BITPIX=-64 files with success before (i.e 64-bit floating point) but not 64-bit integers.

[veggiesaurus]

I'm not 100% sure this is just a BITPIX issue. Our underlying CASA FITS parser spits out an error status BADBEGIN. However, I suspect it will also have trouble because the casacore source code suggests it supports float, double, int32, int16 and boolean, but not int64. Is there a particular need for int64 support?

[drtobybrown]

I'm not sure I'm following this discussion 100% so let me know if you need me to do anything. I'm surprised about the issue with the file, it's just a casa image converted to .fits using exportfits and, as I mentioned above, python and DS9 handle it ok. I think a more illuminating error message would also be helpful in helping the user diagnose the problem.

[veggiesaurus]

I think a more illuminating error message would also be helpful in helping the user diagnose the problem.

Agreed :see_no_evil: We should definitely provide more useful error messages when loading images via URL parameters or commandline arguments (I presume you're using one of those methods?)

@pford Do you know if there's any particular reason why casacore doesn't support 64-bit integer FITS images?

[drtobybrown]

Yep, I was loading via command line. However, my reason for doing that is that I couldn't see the file in the file browser...

[veggiesaurus]

Yep, I was loading via command line. However, my reason for doing that is that I couldn't see the file in the file browser...

Right, because currently we attempt to parse FITS files when we list them, an approach which has also lead to the very slow loading times. For 1.5 this behaviour will be adjusted. Is your CASA image 64-bit integer as well? Would you mind emailing me that as well?

[drtobybrown]

It came as a FITS from a collaborator, so I don't have it. Thinking about it about, I'm now not certain it's a converted casa image, rather a mask of a casa image. I'll try to find out more and get back to you