[BUG] Cubeviz: CALIFA datacube "successfully loaded" but doesn't show up in viewers

[PatrickOgle]

Workaround: https://github.com/spacetelescope/jdaviz/issues/629#issuecomment-1009163354

---

CALIFA (optical IFU) datacubes do not show up in viewers.

To Reproduce Steps to reproduce the behavior:

1. Retrieve CALIFA data cube "'NGC5635.V500.rscube.fits" from https://califaserv.caha.es/CALIFA_WEB/public_html/?q=content/califa-explorer-v01&califaid=750
2. Launch CubeViz app in jupyter notebook
3. Load data from notebook cell: cubeviz.app.load_data('NGC5635.V500.rscube.fits')
4. See "Data loaded successfully" message, but nothing appears in the viewers or in their Data dropdown menus.

Expected behavior 1) Should see "Error: data not loaded" if the datafile can't be loaded properly 2) Better yet, fix the loader/add a new loader that can handle this data set (DS9 can!)

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: OSX
• Browser: safari
• Jupyter: [run "jupyter --version"] jupyter core : 4.6.3 jupyter-notebook : 6.0.3 qtconsole : 4.7.4 ipython : 7.14.0 ipykernel : 5.3.0 jupyter client : 6.1.11 jupyter lab : not installed nbconvert : 6.0.7 ipywidgets : 7.5.1 nbformat : 5.0.6 traitlets : 5.0.5

Package versions (please complete the following information):

macOS-10.13.6-x86_64-i386-64bit Python 3.8.2 | packaged by conda-forge | (default, Apr 24 2020, 07:56:27) [Clang 9.0.1 ] Numpy 1.20.2 astropy 4.2.1 specutils 1.3.dev42+g57dd2b28 spectral-cube 0.5.0 pyyaml 5.4.1 click 7.1.2 asteval 0.9.23 idna 2.10 traitlets 5.0.5 bqplot 0.12.26 bqplot-image-gl 1.3.3 glue-core 1.0.1 glue-jupyter 0.5 glue-astronomy 0.1 echo 0.5 ipyvue 1.5.0 ipyvuetify 1.6.1 ipysplitpanes 0.1.0 ipygoldenlayout 0.3.0 voila 0.2.6 vispy 0.6.6 Jdaviz 1.2.dev153+g39e1d9a

Additional context (e.g. data files) Add any other context about the problem here. "'NGC5635.V500.rscube.fits" from https://califaserv.caha.es/CALIFA_WEB/public_html/?q=content/califa-explorer-v01&califaid=750

🐱

[pllim]

FWIW, more info below:

Their HTTPS gave me TLS warnings. Direct data download link from that page is ftp://ftp.caha.es/CALIFA/reduced/V500/reduced_v2.2/NGC5635.V500.rscube.fits.gz (78 MB) .

Filename: NGC5635.V500.rscube.fits.gz
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU     540   (77, 72, 1877)   float32   
  1  ERROR         1 ImageHDU         9   (77, 72, 1877)   float32   
  2  ERRWEIGHT     1 ImageHDU         9   (77, 72, 1877)   float32   
  3  BADPIX        1 ImageHDU         9   (77, 72, 1877)   uint8   
  4  FIBCOVER      1 ImageHDU         9   (77, 72, 1877)   uint8   
  5  FLAT          1 ImageHDU        40   (77, 72)   float32   

[pllim]

Also, Ginga seems to be able to load it as well, though I see a lot of such warnings from different keywords:

WARNING: VerifyWarning: Keyword name '...' is greater than 8 characters or
contains characters not allowed by the FITS standard; a HIERARCH card will be created.
[astropy.io.fits.card]

[pllim]

Two problems here:

1. Data has invalid CTYPE3 value. There is nothing we can do about this, data provider has to fix it.
2. Cubeviz parser does not recognize the fact that PRIMARY extension can contain science data and the API is not flexible enough for users to overwrite this manually. Something we can fix but PO needs to decide on what API change is desired.

To get this data loaded without any changes to the existing Cubeviz parser code, do the following:

from jdaviz import Cubeviz
from astropy.io import fits

viz = Cubeviz()
fn = 'NGC5635.V500.rscube.fits.gz'
pf = fits.open(fn)
pf[0].header['CTYPE3'] = 'WAVE'  # Fix the first problem
pf[0].name = 'SCI'  # Fix the second problem
viz.load_data(pf)
viz.app

If the provided workaround is good enough for you, this issue can be closed without any further action.

[havok2063]

Does having data in the PRIMARY extension break any sort of (Astropy) FITS standards / verification rules? Also, just pointing that in your above fix, the error and quality viewers are empty, which happens / has happened with our JWST cubes. Although here's it's probably because their extension names are ERROR and BADPIX(?).

The proper fix would probably be to add a new specutils loader for Califa cubes. But it would be nice to allow users to load on-the-fly cubes more easily. Without thinking about it too hard, perhaps minimally by prompting to provide a mapping between the input extensions and the default parser extensions used by Cubeviz.

[pllim]

it would be nice to allow users to load on-the-fly cubes more easily

I agree, especially given there are no standards between all the different observatories on how the data is presented.

Does having data in the PRIMARY extension break any sort of (Astropy) FITS standards

I don't think so. It is pretty common for people to generate images with only primary extension with the data and no other extension. In fact, astropy.io.fits does not complain if you don't care about the missing checksum:

fitscheck NGC5635.V500.rscube.fits.gz --ignore-missing