spacetelescope / jwst

Python library for science observations from the James Webb Space Telescope
https://jwst-pipeline.readthedocs.io/en/latest/
Other
571 stars 167 forks source link

Level3 image3 failure for "ValueError: cannot convert float NaN to integer" #4700

Closed stscijgbot closed 4 years ago

stscijgbot commented 4 years ago

Issue JP-1362 was created by Hien Tran:

Error was seen during skymatch step on the A-string using DMS 7.4.1 of NIRCam dataset [jw00624-o015_20200302t222108_image3_001|https://pljwdmsweb.stsci.edu/owlgui/logs/directory_view?directory_path=/ifs/archive/ops/jwst/info/owl/logs/owlmgr_jw00624-o015_20200302t222108_image3_001_1583194428.787362] {noformat} 2020-03-03 00:16:02,857 - CRDS - DEBUG - ======================================================================================================================== 2020-03-03 00:16:02,857 - CRDS - DEBUG - getreferences() CRDS version: 7.4.1.2, b7.4.1.2, 010d1ed677989f211cf25148704b9090fe1788b2 2020-03-03 00:16:02,857 - CRDS - DEBUG - getreferences() server: https://jwst-serverless-mode.stsci.edu 2020-03-03 00:16:02,857 - CRDS - DEBUG - getreferences() observatory: jwst 2020-03-03 00:16:02,857 - CRDS - DEBUG - getreferences() reftypes: ('drizpars',) 2020-03-03 00:16:02,857 - CRDS - DEBUG - getreferences() context: None 2020-03-03 00:16:02,857 - CRDS - DEBUG - getreferences() ignore_cache: False 2020-03-03 00:16:02,857 - CRDS - DEBUG - CRDS_PATH = /ifs/archive/ops/jwst/ref/tmp_crds/crds/cache 2020-03-03 00:16:02,857 - CRDS - DEBUG - CRDS_SERVER_URL = https://jwst-serverless-mode.stsci.edu 2020-03-03 00:16:02,858 - CRDS - DEBUG - Final effective context is 'jwst_0581.pmap' 2020-03-03 00:16:02,858 - CRDS - DEBUG - Computing best references locally. 2020-03-03 00:16:02,859 - CRDS - DEBUG - Bestrefs header: 2020-03-03 00:16:02,859 - CRDS - DEBUG - {'META.EXPOSURE.READPATT [READPATT]': 'BRIGHT1', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.EXPOSURE.TYPE [EXP_TYPE]': 'NRC_IMAGE', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.INSTRUMENT.CHANNEL [CHANNEL]': 'SHORT', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.INSTRUMENT.CORONAGRAPH [CORONMSK]': 'UNDEFINED', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.INSTRUMENT.DETECTOR [DETECTOR]': 'NRCA1', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.INSTRUMENT.FILTER [FILTER]': 'F070W', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.INSTRUMENT.LAMP_STATE [LAMP]': 'NONE', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.INSTRUMENT.MODULE [MODULE]': 'A', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.INSTRUMENT.NAME [INSTRUME]': 'NIRCAM', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.INSTRUMENT.PUPIL [PUPIL]': 'CLEAR', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.OBSERVATION.DATE [DATE-OBS]': '2020-02-01', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.OBSERVATION.TIME [TIME-OBS]': '02:20:32.071', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.SUBARRAY.NAME [SUBARRAY]': 'FULL', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'META.VISIT.TSOVISIT [TSOVISIT]': 'F', 2020-03-03 00:16:02,859 - CRDS - DEBUG - 'REFTYPE': 'UNDEFINED'} 2020-03-03 00:16:02,859 - CRDS - DEBUG - Reference type 'drizpars' defined as 'jwst_nircam_drizpars_0001.fits' 2020-03-03 00:16:02,859 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/ifs/archive/ops/jwst/ref/tmp_crds/crds/cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'. 2020-03-03 00:16:02,860 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ... 2020-03-03 00:16:18,280 - stpipe.Image3Pipeline - INFO - Aligning input images...

... 2020-03-03 00:18:17,900 - stpipe.Image3Pipeline.tweakreg - INFO - Step tweakreg done 2020-03-03 00:18:17,900 - stpipe.Image3Pipeline - INFO - Matching sky values across all input images... 2020-03-03 00:18:18,336 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch running with args (,). 2020-03-03 00:18:18,337 - stpipe.Image3Pipeline.skymatch - INFO - Step skymatch parameters are: {'pre_hooks': [], 'post_hooks': [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_results': False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/ifs/archive/ops/jwst/info/owlmgr/paths/sdp/asn_creation/cal/level3', 'skymethod': 'global+match', 'match_down': True, 'subtract': False, 'stepsize': None, 'skystat': 'mode', 'dqbits': '0', 'lower': None, 'upper': None, 'nclip': 5, 'lsigma': 4.0, 'usigma': 4.0, 'binwidth': 0.1} 2020-03-03 00:18:20,036 - stpipe.Image3Pipeline.skymatch - INFO - 2020-03-03 00:18:20,036 - stpipe.Image3Pipeline.skymatch - INFO - ***** jwst.skymatch.skymatch.match() started on 2020-03-03 00:18:20.036742 2020-03-03 00:18:20,036 - stpipe.Image3Pipeline.skymatch - INFO - 2020-03-03 00:18:20,036 - stpipe.Image3Pipeline.skymatch - INFO - Sky computation method: 'global+match' 2020-03-03 00:18:20,037 - stpipe.Image3Pipeline.skymatch - INFO - Sky matching direction: DOWN 2020-03-03 00:18:20,037 - stpipe.Image3Pipeline.skymatch - INFO - Sky subtraction from image data: OFF 2020-03-03 00:18:20,038 - stpipe.Image3Pipeline.skymatch - INFO - 2020-03-03 00:18:20,038 - stpipe.Image3Pipeline.skymatch - INFO - ---- Computing differences in sky values in overlapping regions. 2020-03-03 00:18:52,803 - stpipe.Image3Pipeline.skymatch - WARNING - /dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/skymatch/region.py:386: RuntimeWarning: invalid value encountered in true_divide return np.cross(v, w) / D * u + self._start Traceback (most recent call last): File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/bin/strun", line 26, in step = Step.from_cmdline(sys.argv[1:]) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/stpipe/step.py", line 155, in from_cmdline return cmdline.step_from_cmdline(args) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/stpipe/cmdline.py", line 338, in step_from_cmdline step.run(*positional) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/stpipe/step.py", line 391, in run step_result = self.process(*args) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/pipeline/calwebb_image3.py", line 73, in process input_models = self.skymatch(input_models) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/stpipe/step.py", line 391, in run step_result = self.process(*args) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/skymatch/skymatch_step.py", line 97, in process subtract=self.subtract) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/skymatch/skymatch.py", line 312, in match sky_deltas = _find_optimum_sky_deltas(images, apply_sky=not subtract) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/skymatch/skymatch.py", line 511, in _find_optimum_sky_deltas A, W = _overlap_matrix(images, apply_sky=apply_sky) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/skymatch/skymatch.py", line 491, in _overlap_matrix overlap=images[j], delta=apply_sky File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/skymatch/skyimage.py", line 827, in calc_sky sky, npix, area1 = image.calc_sky(overlap=overlap, delta=delta) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/skymatch/skyimage.py", line 446, in calc_sky fill_mask = polygon.scan(fill_mask) File "/dms/local/jwst/pipeline/pkgs/miniconda3/envs/jwstdp-0.14.2-py37/lib/python3.7/site-packages/jwst/skymatch/region.py", line 237, in scan xstart = max(0, int(i) + self._shiftx) ValueError: cannot convert float NaN to integer set savestatus = 1 {noformat}
stscijgbot commented 4 years ago

Comment by Howard Bushouse: [~htran] Please indicate where the data sets can be found, in order for us to triage the problem.

stscijgbot commented 4 years ago

Comment by Catherine Kaleida: We will be running the nOps7 data on the D-string, at which point [~bushouse] will investigate.

stscijgbot commented 4 years ago

Comment by Hien Tran: This error is no longer seen in the reprocessing of nops7 data on the A-string with B7.5 

stscijgbot commented 4 years ago

Comment by Howard Bushouse: Thanks [~htran] for the report. Will close the ticket, since this seems to have been resolved in B7.5.