stscieisenhamer
commented
6 years ago Resources
- Data is located in
/grp/jwst/ssb/eisenhamer/jwst_issues/issue2471
Closed stscieisenhamer closed 5 years ago
stscieisenhamer
commented
6 years ago /grp/jwst/ssb/eisenhamer/jwst_issues/issue2471
hbushouse
commented
6 years ago It looks like the resample step is drizzling 3 separate images, so that suggests multiple inputs to the pipeline. But if there are multiple inputs, then why didn't any of the preceding steps like tweakreg, skymatch, and outlier_detection run? It jumped immediately to resample. And why in the world would it be drizzling 160x160 images into an output frame that's 2087x1153 pixels?
hbushouse
commented
6 years ago Never mind (I should actually look at the data before asking basic questions). Yes, there are 3 input exposures, but it's just 1 each from 3 different detectors (NRCB2, NRCB3, NRCB4), so they don't overlap on the sky. That's why tweakreg, skymatch, and outlier_detection get skipped, and also why the output frame is so big (because it's producing a mosaic of side-by-side images).
jdavies-st
commented
6 years ago So this is failing on SourceCatalogStep. @larrybradley any ideas?
stscieisenhamer
commented
5 years ago Someone fixed something, since it now performs as expected. Note that there is actually no signal in the data, and the pipeline, as seen below, indicates as such:
```
$ strun cfgs/calwebb_image3.cfg jw00609-o003_20180823t161814_image3_001_asn.json
2019-06-12 14:08:01,973 - stpipe - WARNING - /Users/eisenham/miniconda3/envs/jwst-repo/lib/python3.7/site-packages/astropy/modeling/core.py:77: AstropyDeprecationWarning: Composition of model classes will be removed in 4.0 (but composition of model instances is not affected)
AstropyDeprecationWarning)
2019-06-12 14:08:02,191 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created.
2019-06-12 14:08:02,192 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created.
2019-06-12 14:08:02,194 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created.
2019-06-12 14:08:02,195 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created.
2019-06-12 14:08:02,196 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created.
2019-06-12 14:08:02,197 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created.
2019-06-12 14:08:02,200 - stpipe - INFO - Hostname: inanna.local.stsci.edu
2019-06-12 14:08:02,201 - stpipe - INFO - OS: Darwin
2019-06-12 14:08:02,224 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('jw00609-o003_20180823t161814_image3_001_asn.json',).
2019-06-12 14:08:05,079 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb3_cal.fits' reftypes = ['drizpars']
2019-06-12 14:08:06,722 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_nircam_psfmask_0003.rmap 10.4 K bytes (1 / 6 files) (0 / 30.5 K bytes)
2019-06-12 14:08:06,785 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_nircam_dark_0017.rmap 14.2 K bytes (2 / 6 files) (10.4 K / 30.5 K bytes)
2019-06-12 14:08:06,841 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_nircam_0118.imap 2.0 K bytes (3 / 6 files) (24.6 K / 30.5 K bytes)
2019-06-12 14:08:06,891 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_miri_psfmask_0003.rmap 1.3 K bytes (4 / 6 files) (26.6 K / 30.5 K bytes)
2019-06-12 14:08:06,945 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_miri_0175.imap 2.1 K bytes (5 / 6 files) (27.9 K / 30.5 K bytes)
2019-06-12 14:08:07,002 - CRDS - INFO - Fetching /Users/eisenham/Downloads/crds-cache/mappings/jwst/jwst_0534.pmap 573 bytes (6 / 6 files) (29.9 K / 30.5 K bytes)
2019-06-12 14:08:07,222 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'.
2019-06-12 14:08:07,222 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb4_cal.fits' reftypes = ['drizpars']
2019-06-12 14:08:07,225 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'.
2019-06-12 14:08:07,225 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb2_cal.fits' reftypes = ['drizpars']
2019-06-12 14:08:07,227 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'.
2019-06-12 14:08:07,227 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ...
2019-06-12 14:08:08,287 - stpipe.Image3Pipeline - INFO - Resampling images to final result...
2019-06-12 14:08:08,319 - stpipe.Image3Pipeline.resample - INFO - Step resample running with args (
From EMTB DGA#3, running
calwebb_image3on a set of NIRCAMexp_type == "NRC_IMAGE", the following occurs:Show log
``` $ strun cfgs/calwebb_image3.cfg jw00609-o003_20180823t161814_image3_001_asn.json 2018-09-06 14:28:39,528 - stpipe - WARNING - /Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/core.py:71: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. def __init__(self, sigma_clip=SigmaClip(sigma=3., iters=5)): 2018-09-06 14:28:39,528 - stpipe - WARNING - /Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/core.py:114: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. def __init__(self, sigma_clip=SigmaClip(sigma=3., iters=5)): 2018-09-06 14:28:39,530 - stpipe - WARNING - /Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/photutils/background/background_2d.py:285: AstropyDeprecationWarning: "iters" was deprecated in version 3.1 and will be removed in a future version. Use argument "maxiters" instead. sigma_clip=SigmaClip(sigma=3., iters=10), 2018-09-06 14:28:39,634 - stpipe.Image3Pipeline - INFO - Image3Pipeline instance created. 2018-09-06 14:28:39,636 - stpipe.Image3Pipeline.tweakreg - INFO - TweakRegStep instance created. 2018-09-06 14:28:39,637 - stpipe.Image3Pipeline.skymatch - INFO - SkyMatchStep instance created. 2018-09-06 14:28:39,639 - stpipe.Image3Pipeline.outlier_detection - INFO - OutlierDetectionStep instance created. 2018-09-06 14:28:39,640 - stpipe.Image3Pipeline.resample - INFO - ResampleStep instance created. 2018-09-06 14:28:39,641 - stpipe.Image3Pipeline.source_catalog - INFO - SourceCatalogStep instance created. 2018-09-06 14:28:39,643 - stpipe - INFO - Hostname: inanna2.stsci.edu 2018-09-06 14:28:39,643 - stpipe - INFO - OS: Darwin 2018-09-06 14:28:39,679 - stpipe.Image3Pipeline - INFO - Step Image3Pipeline running with args ('jw00609-o003_20180823t161814_image3_001_asn.json',). 2018-09-06 14:28:40,284 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb3_cal.fits' reftypes = ['drizpars'] 2018-09-06 14:28:40,572 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'. 2018-09-06 14:28:40,910 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb4_cal.fits' reftypes = ['drizpars'] 2018-09-06 14:28:40,914 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'. 2018-09-06 14:28:41,260 - stpipe.Image3Pipeline - INFO - Prefetching reference files for dataset: 'jw00609003001_02102_00001_nrcb2_cal.fits' reftypes = ['drizpars'] 2018-09-06 14:28:41,264 - stpipe.Image3Pipeline - INFO - Prefetch for DRIZPARS reference file is '/Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits'. 2018-09-06 14:28:41,264 - stpipe.Image3Pipeline - INFO - Starting calwebb_image3 ... 2018-09-06 14:28:42,255 - stpipe.Image3Pipeline - INFO - Resampling images to final result... 2018-09-06 14:28:42,294 - stpipe.Image3Pipeline.resample - INFO - Step resample running with args (,).
2018-09-06 14:28:42,300 - stpipe.Image3Pipeline.resample - INFO - Drizpars reference file: /Users/eisenham/Downloads/crds-cache/references/jwst/nircam/jwst_nircam_drizpars_0001.fits
2018-09-06 14:28:42,783 - stpipe.Image3Pipeline.resample - INFO - Blending metadata for jw00609-o003_t001_nircam_f070w-sub160
2018-09-06 14:28:43,995 - stpipe.Image3Pipeline.resample - INFO - Drizzling (160, 160) --> (2087, 1153)
2018-09-06 14:28:44,056 - stpipe.Image3Pipeline.resample - INFO - Drizzling (160, 160) --> (2087, 1153)
2018-09-06 14:28:44,120 - stpipe.Image3Pipeline.resample - INFO - Drizzling (160, 160) --> (2087, 1153)
2018-09-06 14:28:44,146 - stpipe.Image3Pipeline.resample - INFO - Update S_REGION to POLYGON ICRS 275.640512876 65.568137010 275.640547240 65.604702768 275.591636954 65.604702769 275.591671310 65.568137011
2018-09-06 14:28:44,150 - stpipe.Image3Pipeline.resample - INFO - Step resample done
2018-09-06 14:28:45,333 - stpipe.Image3Pipeline - INFO - Saved model in jw00609-o003_t001_nircam_f070w-sub160_i2d.fits
2018-09-06 14:28:45,333 - stpipe.Image3Pipeline - INFO - Creating source catalog...
2018-09-06 14:28:45,393 - stpipe.Image3Pipeline.source_catalog - INFO - Step source_catalog running with args (,).
----------------------------------------------------------------------
ERROR RUNNING STEP 'Image3Pipeline':
output array is read-only
Traceback (most recent call last):
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/stpipe/cmdline.py", line 297, in step_from_cmdline
step.run(*positional)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/stpipe/step.py", line 400, in run
step_result = self.process(*args)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/pipeline/calwebb_image3.py", line 103, in process
out_catalog = self.source_catalog(result)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/stpipe/step.py", line 400, in run
step_result = self.process(*args)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/source_catalog/source_catalog_step.py", line 43, in process
npixels, deblend=deblend)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/jwst-0.10.1a0.dev133+gb7284ab1-py3.6-macosx-10.7-x86_64.egg/jwst/source_catalog/source_catalog.py", line 98, in make_source_catalog
model.data, mask=mask, sigma=3.0, maxiters=10)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/astropy/utils/decorators.py", line 488, in wrapper
return function(*args, **kwargs)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/astropy/stats/sigma_clipping.py", line 749, in sigma_clipped_stats
mean = np.nanmean(data_clipped, axis=axis)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/numpy/lib/nanfunctions.py", line 865, in nanmean
avg = _divide_by_count(tot, cnt, out=out)
File "/Users/eisenham/anaconda3/envs/condadev/lib/python3.6/site-packages/numpy/lib/nanfunctions.py", line 179, in _divide_by_count
return np.divide(a, b, out=a, casting='unsafe')
ValueError: output array is read-only
```