search

kevin218 / Eureka

Eureka! is a data reduction and analysis pipeline intended for time-series observations with JWST.
https://eurekadocs.readthedocs.io/
MIT License
62 stars 48 forks source link

[Bug]: S2 crashes on real NIRSpec data with Eureka3.0 #410

Closed trappist1i closed 2 years ago

trappist1i commented 2 years ago

FAQ check

Instrument

NIRSpec (Stages 1-3)

What happened?

When analyzing real NIRSpec/Prism BOTS data (transit spectroscopy), the new version v0.3 of Eureka crashes at stage 2 with the following error message. The previous version did not show this crash. I also add a screen capture of my ecf file.

S2_ecf_screenshot

Error traceback output

2022-07-19 12:13:29,330 - stpipe.EurekaSpec2Pipeline.extract_2d - INFO - Step extract_2d parameters are: {'pre_hooks': [], 'post_ho [], 'output_file': None, 'output_dir': None, 'output_ext': '.fits', 'output_use_model': False, 'output_use_index': True, 'save_res: False, 'skip': False, 'suffix': None, 'search_output_file': True, 'input_dir': '/Users/clavainl/JWST_REDU/data/Trappist1/Nirspec_Transit1/NRS1', 'slit_name': None, 'extract_orders': None, 'grism_objects': None, 'tsgrism_extract_height': None, 'wfss_extract_halght': 5, 'wfss_mmag_extract': None, 'wfss_nbright': 1000} 2022-07-19 12:13:29,367 - stpipe.EurekaSpec2Pipeline.extract_2d - INFO - EXP_TYPE is NRS_BRIGHTOBJ Traceback (most recent call last): File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/pipeline/calwebb_spec2.py", line 117, in process result = self.process_exposure_product( File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/pipeline/calwebb_spec2.py", line 249, in process_exp_product calibrated = self._process_nirspec_slits(calibrated) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/pipeline/calwebb_spec2.py", line 460, in _process_ni_slits calibrated = self.extract_2d(data) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/stpipe/step.py", line 485, in run step_result = self.process(args) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/extract_2d_step.py", line 37, in process output_model = extract_2d.extract2d(dm, self.slit_name, File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/extract_2d.py", line 69, in extract2d output_model = nrs_extract2d(input_model, slit_name=slit_name) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/nirspec.py", line 53, in nrs_extract2d output_model, xlo, xhi, ylo, yhi = process_slit(input_model, slit, exp_type) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/nirspec.py", line 118, in process_slit new_model, xlo, xhi, ylo, yhi = extract_slit(input_model, slit, exp_type) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/nirspec.py", line 211, in extract_slit slit_wcs = nirspec.nrs_wcs_set_input(input_model, slit.name) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/assign_wcs/nirspec.py", line 1660, in nrs_wcs_set_in bb = compute_bounding_box(transform, wavelength_range, File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/assign_wcs/nirspec.py", line 1250, in compute_boundix nsteps = int((lam_max - lam_min) / step) TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType' Traceback (most recent call last): File "/Users/clavainl/JWST_REDU/data_analysis/Trappist1/Nirspec_2420/Transit1/run_eureka.py", line 25, in s2_meta = s2.calibrateJWST(eventlabel, ecf_path=ecf_path) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/eureka/S2_calibrations/s2_calibrate.py", line 159, in caleJWST pipeline.run_eurekaS2(filename, meta, log) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/eureka/S2_calibrations/s2_calibrate.py", line 252, in runkaS2 self(filename) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/stpipe/step.py", line 485, in run step_result = self.process(args) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/pipeline/calwebb_spec2.py", line 135, in process raise RuntimeError('\n'.join(failures)) RuntimeError: Traceback (most recent call last): File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/pipeline/calwebb_spec2.py", line 117, in process result = self.process_exposure_product( File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/pipeline/calwebb_spec2.py", line 249, in process_exp_product calibrated = self._process_nirspec_slits(calibrated) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/pipeline/calwebb_spec2.py", line 460, in _process_ni_slits calibrated = self.extract_2d(data) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/stpipe/step.py", line 485, in run step_result = self.process(*args) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/extract_2d_step.py", line 37, in process output_model = extract_2d.extract2d(dm, self.slit_name, File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/extract_2d.py", line 69, in extract2d output_model = nrs_extract2d(input_model, slit_name=slit_name) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/nirspec.py", line 53, in nrs_extract2d output_model, xlo, xhi, ylo, yhi = process_slit(input_model, slit, exp_type) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/nirspec.py", line 118, in process_slit new_model, xlo, xhi, ylo, yhi = extract_slit(input_model, slit, exp_type) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/extract_2d/nirspec.py", line 211, in extract_slit slit_wcs = nirspec.nrs_wcs_set_input(input_model, slit.name) File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/assign_wcs/nirspec.py", line 1660, in nrs_wcs_set_in bb = compute_bounding_box(transform, wavelength_range, File "/Users/clavainl/anaconda2/envs/eureka/lib/python3.9/site-packages/jwst/assign_wcs/nirspec.py", line 1250, in compute_boundix nsteps = int((lam_max - lam_min) / step) TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

What operating system are you using?

Mac OS 10.14.6

What version of Python are you running?

Python 3.9.7

What Python packages do you have installed?

packages in environment at /Users/clavainl/anaconda2/envs/eureka:

#

Name Version Build Channel

alabaster 0.7.12 pypi_0 pypi appnope 0.1.3 pypi_0 pypi asdf 2.12.0 pypi_0 pypi asdf-astropy 0.2.1 pypi_0 pypi asdf-coordinates-schemas 0.1.0 pypi_0 pypi asdf-standard 1.0.2 pypi_0 pypi asdf-transform-schemas 0.2.2 pypi_0 pypi asdf-wcs-schemas 0.1.1 pypi_0 pypi asteval 0.9.27 pypi_0 pypi astraeus 0.2 pypi_0 pypi astropy 5.1 pypi_0 pypi astropy-healpix 0.6 pypi_0 pypi astroquery 0.4.6 pypi_0 pypi astroscrappy 1.1.0 pypi_0 pypi asttokens 2.0.5 pypi_0 pypi attrs 21.4.0 pypi_0 pypi babel 2.10.3 pypi_0 pypi backcall 0.2.0 pypi_0 pypi batman-package 2.4.9 pypi_0 pypi bayesicfitting 3.0.1 pypi_0 pypi beautifulsoup4 4.11.1 pypi_0 pypi bokeh 2.4.3 pypi_0 pypi bottleneck 1.3.5 pypi_0 pypi ca-certificates 2022.4.26 hecd8cb5_0
ccdproc 2.3.1 pypi_0 pypi celerite 0.4.2 pypi_0 pypi certifi 2022.6.15 py39hecd8cb5_0
cftime 1.6.1 pypi_0 pypi charset-normalizer 2.1.0 pypi_0 pypi cloudpickle 2.1.0 pypi_0 pypi corner 2.2.1 pypi_0 pypi crds 11.16.5 pypi_0 pypi cycler 0.11.0 pypi_0 pypi cython 0.29.30 pypi_0 pypi dask 2022.6.0 pypi_0 pypi decorator 5.1.1 pypi_0 pypi docutils 0.18.1 pypi_0 pypi drizzle 1.13.4 pypi_0 pypi dynesty 1.2.3 pypi_0 pypi emcee 3.1.2 pypi_0 pypi eureka 0.3 pypi_0 pypi executing 0.8.3 pypi_0 pypi exotic-ld 2.0.0 pypi_0 pypi filelock 3.7.1 pypi_0 pypi fonttools 4.34.4 pypi_0 pypi fsspec 2022.5.0 pypi_0 pypi future 0.18.2 pypi_0 pypi george 0.4.0 pypi_0 pypi gwcs 0.18.1 pypi_0 pypi h5netcdf 1.0.1 pypi_0 pypi h5py 3.1.0 pypi_0 pypi html5lib 1.1 pypi_0 pypi idna 3.3 pypi_0 pypi imageio 2.19.3 pypi_0 pypi imagesize 1.4.1 pypi_0 pypi importlib-metadata 4.12.0 pypi_0 pypi iniconfig 1.1.1 pypi_0 pypi ipython 8.4.0 pypi_0 pypi jedi 0.18.1 pypi_0 pypi jinja2 3.1.2 pypi_0 pypi jmespath 1.0.1 pypi_0 pypi jsonschema 4.7.2 pypi_0 pypi jwst 1.6.0 pypi_0 pypi keyring 23.7.0 pypi_0 pypi kiwisolver 1.4.4 pypi_0 pypi libcxx 12.0.0 h2f01273_0
libffi 3.3 hb1e8313_2
lmfit 1.0.3 pypi_0 pypi locket 1.0.0 pypi_0 pypi lxml 4.9.1 pypi_0 pypi markupsafe 2.1.1 pypi_0 pypi matplotlib 3.5.2 pypi_0 pypi matplotlib-inline 0.1.3 pypi_0 pypi ncurses 6.3 hca72f7f_3
netcdf4 1.6.0 pypi_0 pypi networkx 2.8.4 pypi_0 pypi numpy 1.23.1 pypi_0 pypi numpydoc 1.4.0 pypi_0 pypi openssl 1.1.1q hca72f7f_0
packaging 21.3 pypi_0 pypi pandas 1.4.3 pypi_0 pypi parsley 1.3 pypi_0 pypi parso 0.8.3 pypi_0 pypi partd 1.2.0 pypi_0 pypi pexpect 4.8.0 pypi_0 pypi photutils 1.5.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 9.2.0 pypi_0 pypi pip 22.1.2 py39hecd8cb5_0
pluggy 1.0.0 pypi_0 pypi poppy 1.0.3 pypi_0 pypi prompt-toolkit 3.0.30 pypi_0 pypi psutil 5.9.1 pypi_0 pypi ptyprocess 0.7.0 pypi_0 pypi pure-eval 0.2.2 pypi_0 pypi py 1.11.0 pypi_0 pypi pyerfa 2.0.0.1 pypi_0 pypi pygments 2.12.0 pypi_0 pypi pyparsing 3.0.9 pypi_0 pypi pyrsistent 0.18.1 pypi_0 pypi pysynphot 2.0.0 pypi_0 pypi pytest 7.1.2 pypi_0 pypi python 3.9.7 h88f2d9e_1
python-dateutil 2.8.2 pypi_0 pypi pytz 2022.1 pypi_0 pypi pyvo 1.3 pypi_0 pypi pywavelets 1.3.0 pypi_0 pypi pyyaml 6.0 pypi_0 pypi readline 8.1.2 hca72f7f_1
reproject 0.8 pypi_0 pypi requests 2.28.1 pypi_0 pypi scikit-image 0.19.3 pypi_0 pypi scipy 1.8.1 pypi_0 pypi semantic-version 2.10.0 pypi_0 pypi setuptools 61.2.0 py39hecd8cb5_0
six 1.16.0 pypi_0 pypi snowballstemmer 2.2.0 pypi_0 pypi soupsieve 2.3.2.post1 pypi_0 pypi spherical-geometry 1.2.22 pypi_0 pypi sphinx 5.0.2 pypi_0 pypi sphinxcontrib-applehelp 1.0.2 pypi_0 pypi sphinxcontrib-devhelp 1.0.2 pypi_0 pypi sphinxcontrib-htmlhelp 2.0.0 pypi_0 pypi sphinxcontrib-jsmath 1.0.1 pypi_0 pypi sphinxcontrib-qthelp 1.0.3 pypi_0 pypi sphinxcontrib-serializinghtml 1.1.5 pypi_0 pypi sqlite 3.38.5 h707629a_0
stack-data 0.3.0 pypi_0 pypi stcal 1.0.0 pypi_0 pypi stdatamodels 0.4.3 pypi_0 pypi stpipe 0.4.1 pypi_0 pypi stsci-image 2.3.5 pypi_0 pypi stsci-imagestats 1.6.3 pypi_0 pypi stsci-stimage 0.2.5 pypi_0 pypi svo-filters 0.4.3 pypi_0 pypi tifffile 2022.5.4 pypi_0 pypi tk 8.6.12 h5d9f67b_0
tomli 2.0.1 pypi_0 pypi toolz 0.12.0 pypi_0 pypi tornado 6.2 pypi_0 pypi tqdm 4.64.0 pypi_0 pypi traitlets 5.3.0 pypi_0 pypi tweakwcs 0.7.4 pypi_0 pypi typing-extensions 4.3.0 pypi_0 pypi tzdata 2022a hda174b7_0
uncertainties 3.1.7 pypi_0 pypi urllib3 1.26.10 pypi_0 pypi wcwidth 0.2.5 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi wheel 0.37.1 pyhd3eb1b0_0
wiimatch 0.2.1 pypi_0 pypi xarray 2022.3.0 pypi_0 pypi xz 5.2.5 hca72f7f_1
zipp 3.8.1 pypi_0 pypi zlib 1.2.12 h4dc903c_2

Code of Conduct

AarynnCarter commented 2 years ago

@trappist1i This is a bug on my part, the values of None for the waverange_start / waverange_end won't work. Fortunately, the dispersion extraction issue has been fixed.

waverange_start = 6e-08
waverange_end = 6e-06

Should get things running.

For developers: This is an easy fix, we just need to add a catch somewhere that if these values are None, then replace them with the values I defined above.

taylorbell57 commented 2 years ago

@trappist1i, once you have done what Aarynn suggested above, I suspect your Stage 3 should also work, resolving #402

trappist1i commented 2 years ago

Thanks, Aarynn, it solved the issue! But it did not solve my issue with Stage 3 (https://github.com/kevin218/Eureka/issues/402)