search

spacetelescope / stwcs

WCS based distortion models and coordinate transformation.
http://stwcs.readthedocs.io
4 stars 17 forks source link

altwcs now causes pipeline software to crash #144

Closed stsci-hack closed 4 years ago

stsci-hack commented 4 years ago

Some change after commit b7c7aaa has resulted in the pipeline software 'runastrodriz' to crash with the following traceback:

WARNING: wcs_from_key: Could not read WCS with key A [stwcs.wcsutil.altwcs]
##############################################################################
#                                                                            #
# ERROR:                                                                     #
# AstroDrizzle Version svm0.3.1.dev9+g8b7ab5a5 encountered a problem!        #
# Processing terminated at 13:49:08.882 (02/07/2020).                        #
#                                                                            #
##############################################################################
Traceback (most recent call last):
  File "/user/hack/dev/release/stsci_python/pypi/stwcs/git/stwcs/stwcs/wcsutil/altwcs.py", line 525, in wcs_from_key
    w = pywcs.WCS(hdr, fobj=fobj, key=from_key)
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/wcs/wcs.py", line 449, in __init__
    header, fobj, dist='CPDIS', err=minerr)
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/wcs/wcs.py", line 946, in _read_distortion_kw
    d_data = fobj['WCSDVARR', d_extver].data
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/io/fits/hdu/hdulist.py", line 307, in __getitem__
    self._positive_index_of(key))
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/io/fits/hdu/hdulist.py", line 761, in _positive_index_of
    index = self.index_of(key)
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/io/fits/hdu/hdulist.py", line 742, in index_of
    raise KeyError(f'Extension {key!r} not found.')
KeyError: "Extension ('WCSDVARR', 1.0) not found."

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/util.py", line 225, in wrapper
    result = func(*args, **kwargs)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/astrodrizzle.py", line 219, in run
    procSteps=procSteps)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/ablot.py", line 186, in runBlot
    wcsmap=wcsmap)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/ablot.py", line 318, in run_blot
    virtual=img.inmemory)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/outputimage.py", line 455, in writeFITS
    wcs_functions.removeAllAltWCS(fo,wcs_ext)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/wcs_functions.py", line 497, in removeAllAltWCS
    altwcs.deleteWCS(hdulist,extn,wkey)
  File "/user/hack/dev/release/stsci_python/pypi/stwcs/git/stwcs/stwcs/wcsutil/altwcs.py", line 358, in deleteWCS
    hwcs = wcs_from_key(fobj, i, from_key=wkey)
  File "/user/hack/dev/release/stsci_python/pypi/stwcs/git/stwcs/stwcs/wcsutil/altwcs.py", line 528, in wcs_from_key
    log.warning(f'              Skipping {fobj.filename():s}[{ext}]')
TypeError: unsupported format string passed to NoneType.__format__
ERROR: Could not complete astrodrizzle processing of j8c107010_pipeline_asn.fits.
unsupported format string passed to NoneType.__format__
ERROR: Cannot run astrodrizzle on -id.
Traceback (most recent call last):
  File "/user/hack/dev/release/stsci_python/pypi/stwcs/git/stwcs/stwcs/wcsutil/altwcs.py", line 525, in wcs_from_key
    w = pywcs.WCS(hdr, fobj=fobj, key=from_key)
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/wcs/wcs.py", line 449, in __init__
    header, fobj, dist='CPDIS', err=minerr)
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/wcs/wcs.py", line 946, in _read_distortion_kw
    d_data = fobj['WCSDVARR', d_extver].data
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/io/fits/hdu/hdulist.py", line 307, in __getitem__
    self._positive_index_of(key))
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/io/fits/hdu/hdulist.py", line 761, in _positive_index_of
    index = self.index_of(key)
  File "/user/hack/dev/miniconda3/envs/rthstdev/lib/python3.6/site-packages/astropy/io/fits/hdu/hdulist.py", line 742, in index_of
    raise KeyError(f'Extension {key!r} not found.')
KeyError: "Extension ('WCSDVARR', 1.0) not found."

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/runastrodriz.py", line 716, in run_driz
    **pipeline_pars)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/astrodrizzle.py", line 118, in AstroDrizzle
    run(configObj, wcsmap=wcsmap)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/util.py", line 239, in wrapper
    raise errorobj
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/util.py", line 225, in wrapper
    result = func(*args, **kwargs)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/astrodrizzle.py", line 219, in run
    procSteps=procSteps)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/ablot.py", line 186, in runBlot
    wcsmap=wcsmap)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/ablot.py", line 318, in run_blot
    virtual=img.inmemory)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/outputimage.py", line 455, in writeFITS
    wcs_functions.removeAllAltWCS(fo,wcs_ext)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/wcs_functions.py", line 497, in removeAllAltWCS
    altwcs.deleteWCS(hdulist,extn,wkey)
  File "/user/hack/dev/release/stsci_python/pypi/stwcs/git/stwcs/stwcs/wcsutil/altwcs.py", line 358, in deleteWCS
    hwcs = wcs_from_key(fobj, i, from_key=wkey)
  File "/user/hack/dev/release/stsci_python/pypi/stwcs/git/stwcs/stwcs/wcsutil/altwcs.py", line 528, in wcs_from_key
    log.warning(f'              Skipping {fobj.filename():s}[{ext}]')
TypeError: unsupported format string passed to NoneType.__format__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/runastrodriz.py", line 1356, in main
    align_to_gaia=align_to_gaia, force_alignment=force_alignment, debug=debug)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/runastrodriz.py", line 438, in process
    find_crs=True, **adriz_pars)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/runastrodriz.py", line 942, in verify_alignment
    good_bits=good_bits, **pipeline_pars)
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/runastrodriz.py", line 729, in run_driz
    raise Exception(str(errorobj))
Exception: unsupported format string passed to NoneType.__format__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/user/hack/dev/miniconda3/envs/rthstdev/bin/runastrodriz", line 11, in <module>
    load_entry_point('drizzlepac', 'console_scripts', 'runastrodriz')()
  File "/user/hack/dev/release/stsci_python/pypi/drizzlepac/git/drizzlepac/drizzlepac/runastrodriz.py", line 1361, in main
    raise Exception(str(errorobj))
Exception: unsupported format string passed to NoneType.__format__

This problem can be reproduced with the following commands in an ipython session:

from drizzlepac.haputils import astroquery_utils as aqutils
files = aqutils.retrieve_observation(['j8c107010'], suffix=['FLT', 'FLC', 'RAW', 'ASN'], clobber=True) 
!runastrodriz -id j8c107010_asn.fits
stsci-hack commented 4 years ago

Experiments indicate that this error was introduced when #141 was merged in as commit ef9ed9d.

mcara commented 4 years ago

I am not sure, but it is very possible that https://github.com/spacetelescope/stwcs/pull/148 may fix this issue. CC: @nden

mcara commented 4 years ago

@stsci-hack Could you, please, re-test with the latest stwcs master?

stsci-hack commented 4 years ago

This problem still shows up when running runastrodriz while using stwcs version 1.5.4a1.dev44+g4a57476 (master as of 7-Jul-2020).

mcara commented 4 years ago

Experiments indicate that this error was introduced when #141 was merged in as commit ef9ed9d.

The bug is actually in change from '%s' % None (which worked with None) to f'{fobj.filename():s}' which crashes when filename() is None which happens when fobj is an in-memory HDUList. This bug was introduced in:

https://github.com/spacetelescope/stwcs/pull/132/files#diff-516dcd10bb4c104d97362626d506e981L521-R521