asdf-format / asdf

ASDF (Advanced Scientific Data Format) is a next generation interchange format for scientific data
http://asdf.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
524 stars 57 forks source link

asdf.open doesn't handle open HDUList objects, but AsdfInFits.open does #1007

Closed jdavies-st closed 1 year ago

jdavies-st commented 3 years ago

asdf.open() barfs when passed an open HDUList. See the following:

In [5]: with fits.open("jw42424001001_01101_00001_nrca5_cal.fits") as hdulist:
   ...:     with asdf.open(hdulist) as af:
   ...:         print(af.info())
   ...: 
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-5-40505fdfed1e> in <module>
      1 with fits.open("jw42424001001_01101_00001_nrca5_cal.fits", memmap=False) as hdulist:
----> 2     with asdf.open(hdulist) as af:
      3         print(af.info())
      4 

~/miniconda3/envs/jwst/lib/python3.9/site-packages/asdf/asdf.py in open_asdf(fd, uri, mode, validate_checksums, extensions, ignore_version_mismatch, ignore_unrecognized_tag, _force_raw_types, copy_arrays, lazy_load, custom_schema, strict_extension_check, ignore_missing_extensions, _compat, **kwargs)
   1721                    custom_schema=custom_schema, _readonly=readonly)
   1722 
-> 1723     return AsdfFile._open_impl(instance,
   1724         fd, uri=uri, mode=mode,
   1725         validate_checksums=validate_checksums,

~/miniconda3/envs/jwst/lib/python3.9/site-packages/asdf/asdf.py in _open_impl(cls, self, fd, uri, mode, validate_checksums, extensions, _get_yaml_content, _force_raw_types, strict_extension_check, ignore_missing_extensions, **kwargs)
    904                    **kwargs):
    905         """Attempt to open file-like object as either AsdfFile or AsdfInFits"""
--> 906         generic_file = generic_io.get_file(fd, mode=mode, uri=uri)
    907         file_type = util.get_file_type(generic_file)
    908 

~/miniconda3/envs/jwst/lib/python3.9/site-packages/asdf/generic_io.py in get_file(init, mode, uri, close)
   1060         return InputStream(init, mode, uri=uri, close=close)
   1061 
-> 1062     raise ValueError("Can't handle '{0}' as a file for mode '{1}'".format(
   1063         init, mode))

ValueError: Can't handle '[<astropy.io.fits.hdu.image.PrimaryHDU object at 0x7febf01f1670>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7febc04549d0>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7febe2209e20>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7febc03cae80>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7febc0489100>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7febc04893a0>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7febc04895e0>, <astropy.io.fits.hdu.image.ImageHDU object at 0x7febc0489820>, <astropy.io.fits.hdu.table.BinTableHDU object at 0x7febc0489a60>]' as a file for mode 'r'

But if one calls AsdfInFits.open() directly:

In [13]: with fits.open("jw42424001001_01101_00001_nrca5_cal.fits") as hdulist:
    ...:     with AsdfInFits.open(hdulist) as af:
    ...:         print(af.info())
    ...: 
root (AsdfObject)
├─asdf_library (Software)
│ ├─author (str): The ASDF Developers
│ ├─homepage (str): http://github.com/asdf-format/asdf
│ ├─name (str): asdf
│ └─version (str): 2.8.0
├─history (dict)
│ └─extensions (list) ...
├─_fits_hash (str): 901e33788ba40fb05458c66d31b32de5b4269524c50fe9703777f481d53862d2
├─area (NDArrayType): shape=(2048, 2048), dtype=float32
├─data (NDArrayType): shape=(2048, 2048), dtype=float32
├─dq (NDArrayType): shape=(2048, 2048), dtype=uint32
├─err (NDArrayType): shape=(2048, 2048), dtype=float32
├─meta (dict)
│ ├─asn (dict) ...
│ ├─background (dict)
│ ├─bunit_data (str): MJy/sr
│ ├─bunit_err (str): MJy/sr
│ ├─cal_step (dict) ...
│ └─24 not shown
├─var_flat (NDArrayType): shape=(2048, 2048), dtype=float32
├─var_poisson (NDArrayType): shape=(2048, 2048), dtype=float32
└─var_rnoise (NDArrayType): shape=(2048, 2048), dtype=float32
Some nodes not shown.

Success. There should probably be some consistency in open() method interface, and hopefully asdf.open() might support passing along an open HDUList instance to AsdfInFits.open() or where ever the heavy lifting is actually done?

Issue seen in spacetelescope/jdaviz#726

braingram commented 1 year ago

Closing as AsdfInFits support has moved to stdatamodels: https://github.com/braingram/asdf/blob/4bdaf70891cf61e025fd2ea93f56d75e8132eca7/asdf/_tests/_issues/test_1013.py