search

spacetelescope / jwst

Python library for science observations from the James Webb Space Telescope
https://jwst-pipeline.readthedocs.io/en/latest/
Other
565 stars 167 forks source link

MIR_IMAGE + SUBPRISM: zero-size array to reduction operation minimum which has no identity #2426

Closed stscieisenhamer closed 6 years ago

stscieisenhamer commented 6 years ago

Issue

From newly minted data generated during an EMTB observation run, the following occurs when trying to run calwebb_image3 on MIR_IMAGE SUBPRISM.

Show log

``` $ strun ../cfgs/calwebb_image3.cfg jw00608-o002_20180823t163129_image3_001_asn.json 2018-08-27 03:34:09,984 - stpipe - WARNING - /ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/core.py:71: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. def __init__(self, sigma_clip=SigmaClip(sigma=3., iters=5)): 2018-08-27 03:34:09,984 - stpipe - WARNING - /ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/core.py:114: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. def __init__(self, sigma_clip=SigmaClip(sigma=3., iters=5)): 2018-08-27 03:34:09,989 - stpipe - WARNING - /ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/background_2d.py:285: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. sigma_clip=SigmaClip(sigma=3., iters=10), 2018-08-27 03:34:10,609 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created. 2018-08-27 03:34:10,611 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created. 2018-08-27 03:34:10,613 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created. 2018-08-27 03:34:10,614 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created. 2018-08-27 03:34:10,616 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created. 2018-08-27 03:34:10,617 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created. 2018-08-27 03:34:10,617 - stpipe - INFO - Hostname: iljwdmscdp1.stsci.edu 2018-08-27 03:34:10,618 - stpipe - INFO - OS: Linux 2018-08-27 03:34:10,652 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('jw00608-o002_20180823t163129_image3_001_asn.json',). 2018-08-27 03:34:11,426 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00608002001_02101_00001_mirimage_cal.fits' reftypes = ['drizpars'] 2018-08-27 03:34:11,725 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/grp/crds/cache/references/jwst/jwst_miri_drizpars_0001.fits'. 2018-08-27 03:34:12,179 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00608002001_02101_00002_mirimage_cal.fits' reftypes = ['drizpars'] 2018-08-27 03:34:12,182 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/grp/crds/cache/references/jwst/jwst_miri_drizpars_0001.fits'. 2018-08-27 03:34:12,638 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00608002001_02101_00003_mirimage_cal.fits' reftypes = ['drizpars'] 2018-08-27 03:34:12,642 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/grp/crds/cache/references/jwst/jwst_miri_drizpars_0001.fits'. 2018-08-27 03:34:12,643 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ... 2018-08-27 03:34:14,050 - stpipe.Image3Pipeline - INFO - Aligning input images... 2018-08-27 03:34:14,090 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg running with args (,). 2018-08-27 03:34:14,090 - stpipe.Image3Pipeline.tweakreg - INFO - Step skipped. 2018-08-27 03:34:14,092 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg done 2018-08-27 03:34:14,092 - stpipe.Image3Pipeline - INFO - Matching sky values across all input images... 2018-08-27 03:34:14,123 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch running with args (,). 2018-08-27 03:34:14,301 - stpipe.Image3Pipeline.skymatch - INFO - 2018-08-27 03:34:14,301 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() started on 2018-08-27 03:34:14.301743 2018-08-27 03:34:14,301 - stpipe.Image3Pipeline.skymatch - INFO - 2018-08-27 03:34:14,302 - stpipe.Image3Pipeline.skymatch - INFO - Sky computation method: 'global+match' 2018-08-27 03:34:14,302 - stpipe.Image3Pipeline.skymatch - INFO - Sky matching direction: DOWN 2018-08-27 03:34:14,302 - stpipe.Image3Pipeline.skymatch - INFO - Sky subtraction from image data: OFF 2018-08-27 03:34:14,302 - stpipe.Image3Pipeline.skymatch - INFO - 2018-08-27 03:34:14,302 - stpipe.Image3Pipeline.skymatch - INFO - ---- Computing differences in sky values in overlapping regions. 2018-08-27 03:34:14,967 - stpipe.Image3Pipeline.skymatch - WARNING - Unable to compute sky: No valid data in common image areas ---------------------------------------------------------------------- ERROR RUNNING STEP 'Image3Pipeline': zero-size array to reduction operation minimum which has no identity Traceback (most recent call last): File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/stpipe/cmdline.py", line 297, in step_from_cmdline step.run(*positional) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/stpipe/step.py", line 400, in run step_result = self.process(*args) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/pipeline/calwebb_image3.py", line 73, in process input_models = self.skymatch(input_models) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/stpipe/step.py", line 400, in run step_result = self.process(*args) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/skymatch/skymatch_step.py", line 99, in process subtract=self.subtract) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/skymatch/skymatch.py", line 317, in match refsky = np.amin(sky_deltas[sky_good]) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 2442, in amin initial=initial) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 83, in _wrapreduction return ufunc.reduce(obj, axis, dtype, out, **passkwargs) ValueError: zero-size array to reduction operation minimum which has no identity ---------------------------------------------------------------------- Traceback (most recent call last): File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/EGG-INFO/scripts/strun", line 14, in step = Step.from_cmdline(sys.argv[1:]) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/stpipe/step.py", line 162, in from_cmdline return cmdline.step_from_cmdline(args) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/stpipe/cmdline.py", line 297, in step_from_cmdline step.run(*positional) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/stpipe/step.py", line 400, in run step_result = self.process(*args) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/pipeline/calwebb_image3.py", line 73, in process input_models = self.skymatch(input_models) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/stpipe/step.py", line 400, in run step_result = self.process(*args) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/skymatch/skymatch_step.py", line 99, in process subtract=self.subtract) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev189+gf656b25f-py3.6-linux-x86_64.egg/jwst/skymatch/skymatch.py", line 317, in match refsky = np.amin(sky_deltas[sky_good]) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 2442, in amin initial=initial) File "/ifs/int/jwstc/info/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 83, in _wrapreduction return ufunc.reduce(obj, axis, dtype, out, **passkwargs) ValueError: zero-size array to reduction operation minimum which has no identity ```

stscieisenhamer commented 6 years ago

Resources

hbushouse commented 6 years ago

The SUBPRISM subarray is used exclusively for LRS slitless spectroscopic observations with the P750L prism, so why is it being run through calwebb_image3? It should be processed with calwebb_spec3. Perhaps the DMS "recipes" that are used to determine which data go through which pipeline module need updating.

stscieisenhamer commented 6 years ago

Couple of items:

First, just tried calwebb_spec3 and it dies for other reasons.

However, more importantly, this is from the template "MIRI External Flat", where the detector is IMAGER, subarray is SLITLESSPRISM, but no other elements in the optical path. The exposures get EXP_TYPE == MIR_IMAGE. Three exposures are taken, but not for dithering/mosaic reasons; this seems to be the default for external flats. First guess is that this data should not get Level3 processing at all.

Just discussed with @jemorrison and bumping up to the team.

hbushouse commented 6 years ago

Agree that level-3 processing is not required. For imaging-mode flats, we typically don't even do level-2b (level-2a only). The EXP_TYPE should better reflect the fact that it's a FLAT.

stscieisenhamer commented 6 years ago

From the 28Aug18 Association meeting, it appears that the wrong EXP_TYPEs have been used by commanding. Follow up by D.Swade is in process. Blocking until then. See DR # 5207.

stscieisenhamer commented 6 years ago

From 18Sep18 DMS Design: SDP will be making a new exp_type for the external flats. Until otherwise informed by the team, this will be handle like a mir_flatimage and mir_flatmrs. Still blocking on resolution of DR # 5207.

stscieisenhamer commented 6 years ago

Exposure types have been decided and #2694 and #2695 will work those new types. Closing.