Closed stscieisenhamer closed 5 years ago
/grp/jwst/ssb/eisenhamer/jwst_issues/issue2471
It looks like the resample
step is drizzling 3 separate images, so that suggests multiple inputs to the pipeline. But if there are multiple inputs, then why didn't any of the preceding steps like tweakreg, skymatch, and outlier_detection run? It jumped immediately to resample. And why in the world would it be drizzling 160x160 images into an output frame that's 2087x1153 pixels?
Never mind (I should actually look at the data before asking basic questions). Yes, there are 3 input exposures, but it's just 1 each from 3 different detectors (NRCB2, NRCB3, NRCB4), so they don't overlap on the sky. That's why tweakreg, skymatch, and outlier_detection get skipped, and also why the output frame is so big (because it's producing a mosaic of side-by-side images).
So this is failing on SourceCatalogStep
. @larrybradley any ideas?
Someone fixed something, since it now performs as expected. Note that there is actually no signal in the data, and the pipeline, as seen below, indicates as such:
```
$ strun cfgs/calwebb_image3.cfg jw00609-o003_20180823t161814_image3_001_asn.json
2019-06-12 14:08:01,973 - stpipe - WARNING - /Users/eisenham/miniconda3/envs/jwst-repo/lib/python3.7/site-packages/astropy/modeling/core.py:77: AstropyDeprecationWarning: Composition of model classes will be removed in 4.0 (but composition of model instances is not affected)
AstropyDeprecationWarning)
2019-06-12 14:08:02,191 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created.
2019-06-12 14:08:02,192 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created.
2019-06-12 14:08:02,194 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created.
2019-06-12 14:08:02,195 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created.
2019-06-12 14:08:02,196 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created.
2019-06-12 14:08:02,197 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created.
2019-06-12 14:08:02,200 - stpipe - INFO - Hostname: inanna.local.stsci.edu
2019-06-12 14:08:02,201 - stpipe - INFO - OS: Darwin
2019-06-12 14:08:02,224 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('jw00609-o003_20180823t161814_image3_001_asn.json',).
2019-06-12 14:08:05,079 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb3_cal.fits' reftypes = ['drizpars']
2019-06-12 14:08:06,722 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_nircam_psfmask_0003.rmap 10.4 K bytes (1 / 6 files) (0 / 30.5 K bytes)
2019-06-12 14:08:06,785 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_nircam_dark_0017.rmap 14.2 K bytes (2 / 6 files) (10.4 K / 30.5 K bytes)
2019-06-12 14:08:06,841 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_nircam_0118.imap 2.0 K bytes (3 / 6 files) (24.6 K / 30.5 K bytes)
2019-06-12 14:08:06,891 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_miri_psfmask_0003.rmap 1.3 K bytes (4 / 6 files) (26.6 K / 30.5 K bytes)
2019-06-12 14:08:06,945 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_miri_0175.imap 2.1 K bytes (5 / 6 files) (27.9 K / 30.5 K bytes)
2019-06-12 14:08:07,002 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_0534.pmap 573 bytes (6 / 6 files) (29.9 K / 30.5 K bytes)
2019-06-12 14:08:07,222 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'.
2019-06-12 14:08:07,222 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb4_cal.fits' reftypes = ['drizpars']
2019-06-12 14:08:07,225 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'.
2019-06-12 14:08:07,225 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb2_cal.fits' reftypes = ['drizpars']
2019-06-12 14:08:07,227 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'.
2019-06-12 14:08:07,227 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ...
2019-06-12 14:08:08,287 - stpipe.Image3Pipeline - INFO - Resampling images to final result...
2019-06-12 14:08:08,319 - stpipe.Image3Pipeline.resample - INFO - Step resample running with args (
From EMTB DGA#3, running
calwebb_image3
on a set of NIRCAMexp_type == "NRC_IMAGE"
, the following occurs:Show log
``` $ strun cfgs/calwebb_image3.cfg jw00609-o003_20180823t161814_image3_001_asn.json 2018-09-06 14:28:39,528 - stpipe - WARNING - /Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/core.py:71: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. def __init__(self, sigma_clip=SigmaClip(sigma=3., iters=5)): 2018-09-06 14:28:39,528 - stpipe - WARNING - /Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/core.py:114: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. def __init__(self, sigma_clip=SigmaClip(sigma=3., iters=5)): 2018-09-06 14:28:39,530 - stpipe - WARNING - /Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/background_2d.py:285: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. sigma_clip=SigmaClip(sigma=3., iters=10), 2018-09-06 14:28:39,634 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created. 2018-09-06 14:28:39,636 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created. 2018-09-06 14:28:39,637 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created. 2018-09-06 14:28:39,639 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created. 2018-09-06 14:28:39,640 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created. 2018-09-06 14:28:39,641 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created. 2018-09-06 14:28:39,643 - stpipe - INFO - Hostname: inanna2.stsci.edu 2018-09-06 14:28:39,643 - stpipe - INFO - OS: Darwin 2018-09-06 14:28:39,679 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('jw00609-o003_20180823t161814_image3_001_asn.json',). 2018-09-06 14:28:40,284 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb3_cal.fits' reftypes = ['drizpars'] 2018-09-06 14:28:40,572 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'. 2018-09-06 14:28:40,910 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb4_cal.fits' reftypes = ['drizpars'] 2018-09-06 14:28:40,914 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'. 2018-09-06 14:28:41,260 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb2_cal.fits' reftypes = ['drizpars'] 2018-09-06 14:28:41,264 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'. 2018-09-06 14:28:41,264 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ... 2018-09-06 14:28:42,255 - stpipe.Image3Pipeline - INFO - Resampling images to final result... 2018-09-06 14:28:42,294 - stpipe.Image3Pipeline.resample - INFO - Step resample running with args (,).
2018-09-06 14:28:42,300 - stpipe.Image3Pipeline.resample - INFO - Drizpars reference file: /Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits
2018-09-06 14:28:42,783 - stpipe.Image3Pipeline.resample - INFO - Blending metadata for jw00609-o003_t001_nircam_f070w-sub160
2018-09-06 14:28:43,995 - stpipe.Image3Pipeline.resample - INFO - Drizzling (160, 160) --> (2087, 1153)
2018-09-06 14:28:44,056 - stpipe.Image3Pipeline.resample - INFO - Drizzling (160, 160) --> (2087, 1153)
2018-09-06 14:28:44,120 - stpipe.Image3Pipeline.resample - INFO - Drizzling (160, 160) --> (2087, 1153)
2018-09-06 14:28:44,146 - stpipe.Image3Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 275.640512876 65.568137010 275.640547240 65.604702768 275.591636954 65.604702769 275.591671310 65.568137011
2018-09-06 14:28:44,150 - stpipe.Image3Pipeline.resample - INFO - Step resample done
2018-09-06 14:28:45,333 - stpipe.Image3Pipeline - INFO - Saved model in jw00609-o003_t001_nircam_f070w-sub160_i2d.fits
2018-09-06 14:28:45,333 - stpipe.Image3Pipeline - INFO - Creating source catalog...
2018-09-06 14:28:45,393 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog running with args (,).
----------------------------------------------------------------------
ERROR RUNNING STEP 'Image3Pipeline':
output array is read-only
Traceback (most recent call last):
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/stpipe/cmdline.py", line 297, in step_from_cmdline
step.run(*positional)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/stpipe/step.py", line 400, in run
step_result = self.process(*args)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/pipeline/calwebb_image3.py", line 103, in process
out_catalog = self.source_catalog(result)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/stpipe/step.py", line 400, in run
step_result = self.process(*args)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/source_catalog/source_catalog_step.py", line 43, in process
npixels, deblend=deblend)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/source_catalog/source_catalog.py", line 98, in make_source_catalog
model.data, mask=mask, sigma=3.0, maxiters=10)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/astropy/utils/decorators.py", line 488, in wrapper
return function(*args, **kwargs)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/astropy/stats/sigma_clipping.py", line 749, in sigma_clipped_stats
mean = np.nanmean(data_clipped, axis=axis)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/numpy/lib/nanfunctions.py", line 865, in nanmean
avg = _divide_by_count(tot, cnt, out=out)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/numpy/lib/nanfunctions.py", line 179, in _divide_by_count
return np.divide(a, b, out=a, casting='unsafe')
ValueError: output array is read-only
```