Unexpected Traceback error in stage 5

[amitmazz]

Instrument

Light curve fitting (Stages 4-6)

What happened?

I was running the prism data for wasp39-b. Now I wanted to reproduce the result of the paper 'Early Release Science of the exoplanet WASP-39b with JWST NIRSpec PRISM'. I wanted to get the full spectrum so I put 'None' for the wave_min and wave_max. But I was getting the following error for three different bin sizes of 100,150 and 200. Earlier I thought it is binning which is creating the error. But I ran the pipeline for 100 and 150 bins with specified wave_min and wave_max. So I guess it is the specification of the minimum and maximum wavelength that is causing this (I am not sure exactly). Please shed some light on it. My files can be found in the repository : https://github.com/amitmazz/Eureka_wasp39_b.git

Error traceback output

DYNESTY RESULTS: rp: 0.14559509893344028 (+1.631886845099295e-08, -2.6818667031580645e-07) per: 4.055241161189638 (+8.98391583348257e-10, -4.171418765963608e-11) t0: 57394.46370867514 (+4.18367562815547e-09, -1.2438249541446567e-07) inc: 87.97849108764812 (+5.291561720355276e-06, -9.487919783168763e-09) a: 10.57024646791943 (+6.625779802149623e-06, -2.035343449335869e-08) u1: 0.5144825055126634 (+9.807763823310722e-09, -9.90403740197543e-06) u2: 0.6046437172398047 (+5.587432165499173e-06, -3.647930367778329e-07) c0: 1.0081079442640857 (+2.085072425828116e-08, -1.8893218101112552e-09) c1: -0.004468086669612368 (+8.954250804629055e-08, -2.2118724889858e-09) scatter_mult: 9.309536151601387 (+0.0, -0.0); 8142.200648611712 (+0.0, -0.0) ppm Traceback (most recent call last): File "/home/amit/Eureka_files/JWST/run_eureka.py", line 31, in s5_meta = s5.fitlc(eventlabel, ecf_path=ecf_path, s4_meta=s4_meta) File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/s5_fit.py", line 414, in fitlc meta, params = fit_channel(meta, time, flux, channel, File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/s5_fit.py", line 791, in fit_channel lc_model.fit(model, meta, log, fitter='dynesty') File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/lightcurve.py", line 174, in fit fit_model = self.fitter_func(self, model, meta, log, **kwargs) File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/fitters.py", line 928, in dynestyfitter plots.plot_corner(samples, lc, meta, freenames, fitter='dynesty') File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/plots_s5.py", line 432, in plot_corner fig = corner.corner(samples, fig=fig, quantiles=[0.16, 0.5, 0.84], File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/corner/corner.py", line 256, in corner return arviz_corner( File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/corner/arviz_corner.py", line 136, in arviz_corner fig = corner_impl( File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/corner/core.py", line 121, in corner_impl raise ValueError( ValueError: It looks like the parameter(s) in column(s) 9 have no dynamic range. Please provide a range argument. Exception ignored in: <function CachingFileManager.del at 0x7fa059b4dee0> Traceback (most recent call last): File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/xarray/backends/file_manager.py", line 249, in del File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/xarray/backends/file_manager.py", line 233, in close File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/h5netcdf/core.py", line 1209, in close File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/h5py/_hl/files.py", line 445, in close TypeError: bad operand type for unary ~: 'NoneType' Exception ignored in: <function File.close at 0x7fa1a53d5a60> Traceback (most recent call last): File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/h5netcdf/core.py", line 1209, in close File "/home/amit/anaconda3/envs/eureka/lib/python3.9/site-packages/h5py/_hl/files.py", line 445, in close TypeError: bad operand type for unary ~: 'NoneType'

What operating system are you using?

Ubuntu 20.04.6 LTS

What version of Python are you running?

python 3.9.16

What Python packages do you have installed?

_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
aesara-theano-fallback 0.1.0 pypi_0 pypi alabaster 0.7.13 pypi_0 pypi anyio 3.6.2 pypi_0 pypi argon2-cffi 21.3.0 pypi_0 pypi argon2-cffi-bindings 21.2.0 pypi_0 pypi arrow 1.2.3 pypi_0 pypi arviz 0.12.1 pypi_0 pypi asdf 2.14.4 pypi_0 pypi asdf-astropy 0.4.0 pypi_0 pypi asdf-coordinates-schemas 0.2.0 pypi_0 pypi asdf-standard 1.0.3 pypi_0 pypi asdf-transform-schemas 0.3.0 pypi_0 pypi asdf-unit-schemas 0.1.0 pypi_0 pypi asdf-wcs-schemas 0.1.1 pypi_0 pypi asteval 0.9.29 pypi_0 pypi astraeus 0.3 pypi_0 pypi astropy 5.2.2 pypi_0 pypi astropy-healpix 0.7 pypi_0 pypi astroquery 0.4.6 pypi_0 pypi astroscrappy 1.1.0 pypi_0 pypi asttokens 2.2.1 pypi_0 pypi attrs 22.2.0 pypi_0 pypi babel 2.12.1 pypi_0 pypi backcall 0.2.0 pypi_0 pypi batman-package 2.4.9 pypi_0 pypi bayesicfitting 3.1.1 pypi_0 pypi beautifulsoup4 4.12.0 pypi_0 pypi bleach 6.0.0 pypi_0 pypi bokeh 2.4.3 pypi_0 pypi bottleneck 1.3.7 pypi_0 pypi ca-certificates 2023.01.10 h06a4308_0
cachetools 5.3.0 pypi_0 pypi ccdproc 2.4.0 pypi_0 pypi celerite 0.4.2 pypi_0 pypi certifi 2022.5.18.1 pypi_0 pypi cffi 1.15.1 pypi_0 pypi cftime 1.6.2 pypi_0 pypi charset-normalizer 3.1.0 pypi_0 pypi cloudpickle 2.2.1 pypi_0 pypi codecov 2.1.12 pypi_0 pypi comm 0.1.3 pypi_0 pypi contourpy 1.0.7 pypi_0 pypi corner 2.2.1 pypi_0 pypi coverage 7.2.2 pypi_0 pypi crds 11.16.21 pypi_0 pypi cryptography 40.0.1 pypi_0 pypi cycler 0.11.0 pypi_0 pypi cython 0.29.34 pypi_0 pypi dask 2022.6.0 pypi_0 pypi debugpy 1.6.6 pypi_0 pypi decorator 5.1.1 pypi_0 pypi defusedxml 0.7.1 pypi_0 pypi deprecat 2.1.1 pypi_0 pypi dill 0.3.6 pypi_0 pypi docutils 0.18.1 pypi_0 pypi drizzle 1.13.7 pypi_0 pypi dynesty 2.1.0 pypi_0 pypi emcee 3.1.4 pypi_0 pypi eureka 0.9.dev164+g8808897 pypi_0 pypi exceptiongroup 1.1.1 pypi_0 pypi executing 1.2.0 pypi_0 pypi exoplanet 0.5.3 pypi_0 pypi exoplanet-core 0.1.2 pypi_0 pypi exotic-ld 3.0.0 pypi_0 pypi fastjsonschema 2.16.3 pypi_0 pypi fastprogress 1.0.3 pypi_0 pypi filelock 3.10.7 pypi_0 pypi flake8 6.0.0 pypi_0 pypi fonttools 4.39.3 pypi_0 pypi fqdn 1.5.1 pypi_0 pypi fsspec 2023.3.0 pypi_0 pypi future 0.18.3 pypi_0 pypi george 0.4.0 pypi_0 pypi gwcs 0.18.3 pypi_0 pypi h5netcdf 1.1.0 pypi_0 pypi h5py 3.1.0 pypi_0 pypi html5lib 1.1 pypi_0 pypi idna 3.4 pypi_0 pypi image-registration 0.2.7.dev16+g24e5018 pypi_0 pypi imageio 2.27.0 pypi_0 pypi imagesize 1.4.1 pypi_0 pypi importlib-metadata 6.1.0 pypi_0 pypi importlib-resources 5.12.0 pypi_0 pypi iniconfig 2.0.0 pypi_0 pypi intel-openmp 2021.4.0 h06a4308_3561
ipykernel 6.22.0 pypi_0 pypi ipython 8.12.0 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi ipywidgets 8.0.6 pypi_0 pypi isoduration 20.11.0 pypi_0 pypi jaraco-classes 3.2.3 pypi_0 pypi jedi 0.18.2 pypi_0 pypi jeepney 0.8.0 pypi_0 pypi jinja2 3.1.2 pypi_0 pypi jmespath 1.0.1 pypi_0 pypi jsonpointer 2.3 pypi_0 pypi jsonschema 4.9.1 pypi_0 pypi jupyter 1.0.0 pypi_0 pypi jupyter-client 8.1.0 pypi_0 pypi jupyter-console 6.6.3 pypi_0 pypi jupyter-core 5.3.0 pypi_0 pypi jupyter-events 0.6.3 pypi_0 pypi jupyter-server 2.5.0 pypi_0 pypi jupyter-server-terminals 0.4.4 pypi_0 pypi jupyterlab-pygments 0.2.2 pypi_0 pypi jupyterlab-widgets 3.0.7 pypi_0 pypi jwst 1.8.2 pypi_0 pypi keyring 23.13.1 pypi_0 pypi kiwisolver 1.4.4 pypi_0 pypi lazy-loader 0.2 pypi_0 pypi ld_impl_linux-64 2.38 h1181459_1
libffi 3.4.2 h6a678d5_6
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libstdcxx-ng 11.2.0 h1234567_1
lmfit 1.1.0 pypi_0 pypi locket 1.0.0 pypi_0 pypi lxml 4.9.2 pypi_0 pypi markdown-it-py 2.2.0 pypi_0 pypi markupsafe 2.1.2 pypi_0 pypi matplotlib 3.7.1 pypi_0 pypi matplotlib-inline 0.1.6 pypi_0 pypi mccabe 0.7.0 pypi_0 pypi mdit-py-plugins 0.3.5 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi mistune 2.0.5 pypi_0 pypi mkl 2021.4.0 h06a4308_640
mkl-service 2.4.0 py39h7f8727e_0
more-itertools 9.1.0 pypi_0 pypi myst-parser 1.0.0 pypi_0 pypi nbclassic 0.5.4 pypi_0 pypi nbclient 0.7.3 pypi_0 pypi nbconvert 7.3.0 pypi_0 pypi nbformat 5.8.0 pypi_0 pypi nbsphinx 0.9.1 pypi_0 pypi ncurses 6.4 h6a678d5_0
nest-asyncio 1.5.6 pypi_0 pypi netcdf4 1.6.3 pypi_0 pypi networkx 3.0 pypi_0 pypi notebook 6.5.3 pypi_0 pypi notebook-shim 0.2.2 pypi_0 pypi numpy 1.21.6 pypi_0 pypi numpydoc 1.5.0 pypi_0 pypi opencv-python 4.7.0.72 pypi_0 pypi openssl 1.1.1t h7f8727e_0
packaging 23.0 pypi_0 pypi pandas 1.5.3 pypi_0 pypi pandocfilters 1.5.0 pypi_0 pypi parsley 1.3 pypi_0 pypi parso 0.8.3 pypi_0 pypi partd 1.3.0 pypi_0 pypi patsy 0.5.3 pypi_0 pypi pexpect 4.8.0 pypi_0 pypi photutils 1.6.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 9.5.0 pypi_0 pypi pip 23.0.1 py39h06a4308_0
platformdirs 3.2.0 pypi_0 pypi pluggy 1.0.0 pypi_0 pypi poppy 1.0.3 pypi_0 pypi prometheus-client 0.16.0 pypi_0 pypi prompt-toolkit 3.0.38 pypi_0 pypi psutil 5.9.4 pypi_0 pypi ptyprocess 0.7.0 pypi_0 pypi pure-eval 0.2.2 pypi_0 pypi pycodestyle 2.10.0 pypi_0 pypi pycparser 2.21 pypi_0 pypi pyerfa 2.0.0.3 pypi_0 pypi pyflakes 3.0.1 pypi_0 pypi pygments 2.14.0 pypi_0 pypi pymc3 3.11.5 pypi_0 pypi pymc3-ext 0.1.1 pypi_0 pypi pyparsing 3.0.9 pypi_0 pypi pyrsistent 0.19.3 pypi_0 pypi pysynphot 2.0.0 pypi_0 pypi pytest 7.2.2 pypi_0 pypi pytest-cov 4.0.0 pypi_0 pypi pytest-doctestplus 0.12.1 pypi_0 pypi python 3.9.16 h7a1cb2a_2
python-dateutil 2.8.2 pypi_0 pypi python-json-logger 2.0.7 pypi_0 pypi pytz 2023.3 pypi_0 pypi pyvo 1.4.1 pypi_0 pypi pywavelets 1.4.1 pypi_0 pypi pyyaml 6.0 pypi_0 pypi pyzmq 25.0.2 pypi_0 pypi qtconsole 5.4.2 pypi_0 pypi qtpy 2.3.1 pypi_0 pypi readline 8.2 h5eee18b_0
reproject 0.10.0 pypi_0 pypi requests 2.28.2 pypi_0 pypi rfc3339-validator 0.1.4 pypi_0 pypi rfc3986-validator 0.1.1 pypi_0 pypi scikit-image 0.19.3 pypi_0 pypi scipy 1.7.3 pypi_0 pypi secretstorage 3.3.3 pypi_0 pypi semantic-version 2.10.0 pypi_0 pypi semver 3.0.0 pypi_0 pypi send2trash 1.8.0 pypi_0 pypi setuptools 65.6.3 py39h06a4308_0
setuptools-scm 7.1.0 pypi_0 pypi six 1.16.0 pyhd3eb1b0_1
sniffio 1.3.0 pypi_0 pypi snowballstemmer 2.2.0 pypi_0 pypi soupsieve 2.4 pypi_0 pypi spherical-geometry 1.2.23 pypi_0 pypi sphinx 6.1.3 pypi_0 pypi sphinx-automodapi 0.15.0 pypi_0 pypi sphinx-rtd-theme 1.2.0 pypi_0 pypi sphinxcontrib-applehelp 1.0.4 pypi_0 pypi sphinxcontrib-devhelp 1.0.2 pypi_0 pypi sphinxcontrib-htmlhelp 2.0.1 pypi_0 pypi sphinxcontrib-jquery 4.1 pypi_0 pypi sphinxcontrib-jsmath 1.0.1 pypi_0 pypi sphinxcontrib-qthelp 1.0.3 pypi_0 pypi sphinxcontrib-serializinghtml 1.1.5 pypi_0 pypi sqlite 3.41.1 h5eee18b_0
stack-data 0.6.2 pypi_0 pypi starry 1.2.0 pypi_0 pypi stcal 1.2.2 pypi_0 pypi stdatamodels 0.4.5 pypi_0 pypi stpipe 0.4.6 pypi_0 pypi stsci-image 2.3.5 pypi_0 pypi stsci-imagestats 1.6.3 pypi_0 pypi stsci-stimage 0.2.6 pypi_0 pypi svo-filters 0.4.4 pypi_0 pypi terminado 0.17.1 pypi_0 pypi theano-pymc 1.1.2 pypi_0 pypi tifffile 2023.3.21 pypi_0 pypi tinycss2 1.2.1 pypi_0 pypi tk 8.6.12 h1ccaba5_0
tomli 2.0.1 pypi_0 pypi toolz 0.12.0 pypi_0 pypi tornado 6.2 pypi_0 pypi tqdm 4.65.0 pypi_0 pypi traitlets 5.9.0 pypi_0 pypi tweakwcs 0.8.1 pypi_0 pypi typing-extensions 4.5.0 pypi_0 pypi tzdata 2023c h04d1e81_0
uncertainties 3.1.7 pypi_0 pypi uri-template 1.2.0 pypi_0 pypi urllib3 1.26.15 pypi_0 pypi wcwidth 0.2.6 pypi_0 pypi webcolors 1.13 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi websocket-client 1.5.1 pypi_0 pypi wheel 0.38.4 py39h06a4308_0
widgetsnbextension 4.0.7 pypi_0 pypi wiimatch 0.3.1 pypi_0 pypi wrapt 1.15.0 pypi_0 pypi xarray 2023.3.0 pypi_0 pypi xarray-einstats 0.5.1 pypi_0 pypi xz 5.2.10 h5eee18b_1
zipp 3.15.0 pypi_0 pypi zlib 1.2.13 h5eee18b_0

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[taylorbell57]

Can you please attach the Fig5101 figure that should've been made before this crash happened (make sure it is from a run that suffers from this no dynamic range error message, and not from one of your successful fits)? I need some more contextual information. This happens sometimes with emcee (although I don't think I've ever seen it happen with dynesty) where the sampler won't be run long enough or something else breaks and the uncertainty for one parameter ends up being +/- 0 which is what ultimately caused your crash here. I need to see that Fig 5101 though to understand what about the data may have caused the fitter to break (and then using that, I can hopefully understand what about the code or your ECFs might have caused that).

[amitmazz]

These are the 5101 files that were generated before the crash

[amitmazz]

If any more figures required or any other information please let me know

[kevin218]

You should not adjust the Stage 2 ECF wave_min or wave_max and don't set it to None. The default settings should work just fine.

[amitmazz]

Did you mean Stage 4 ECF? Or did you mean wavelength_start and wavelength_end in stage 2? I did not find wave_min and wave_max in stage 2 ECF. Also I did not exactly understood you, like I should not adjust and also not set it to none, could you please explain ? @kevin218

[kevin218]

Just ignore my last comment. I was thinking of wave start and end in Stage 2.

[amitmazz]

So, How do I tackle the above traceback error?

[taylorbell57]

I'll try to get to this on Monday, but the issue isn't exactly with wave_min and wave_max in Stage 4. If it were, the crash would've happened during Stage 4 or the very first or last channels of Stage 5, not partway through the fits. I've yet to figure out why exactly the scatter_mult failed to sample on that particular channel, but I have some guesses (again, I'll look more closely at this on Monday - I need the weekend break).

[taylorbell57]

I'm looking at this now FYI - there's nothing immediately wrong with your ECFs or EPFs. It's not really a solution and is instead a workaround, but you could switch to using the emcee sampler instead. Another option is to change batman_tr to starry and switch to using the nuts sampler. I'll see if I can reproduce your issue on my machine (I've been meaning to reproduce the PRISM anaysis myself anyway), but my strongest suspicion is that one of your priors is causing the issue (scatter_mult and a are the most likely culprits).

[taylorbell57]

Alright, well I won't be able to finish a reduction today and will be off work tomorrow, so here's some thoughts based on what I've managed to do so far:

1. You shouldn't fit the orbital period and linear ephemeris at the same time since they're degenerate with only one observation. I'd fix per to some literature value while letting t0 vary.
2. Your prior for a is incorrect and is based on a mistake that was made when simulating these observations before the launch of JWST. You should use IPAC's Exoplanet Archive to update your prior. Personally I'd go with the values of Mancini et al. 2018, but you'll need to convert their semi-major axis from AU units to a/R* (while making sure to cancel out units properly). A simpler option is just to use the values from Maciejewski et al. 2016.
3. You should increase the prior range on scatter_mult since some of the lightcurves in the saturated region are going to have really quite high uncertainties. I'd set the PriorPar2 column in the EPF for scatter_mult to 10 just to be minimally informative.
4. You should first fit a white lightcurve with free priors on t0, a, and inc. Then, update the Value column in the EPFs for those 3 parameters to be the results you got from your white lightcurve fit, and set them as 'fixed' since those three orbital parameters should not be allowed to vary freely with wavelength. Then you can go about fitting your spectroscopic lightcurves. (By white lightcurve, I mean set nspecchan to 1 in your S4 ECF).
5. You can significantly decrease your runtime by changing run_nlive to something around 200 to get much faster fits while still getting pretty reliable uncertainties.

Some combination of all of that should resolve the issue you were having with the ValueError: It looks like the parameter(s) in column(s) 9 have no dynamic range. error.

[taylorbell57]

FYI, I've tried reducing and fitting the data myself, and I did not encounter any such issues after taking into account my points above. As I can't reproduce the error accounting for the general takeaways in my message above, I'm going to close this issue, but please do re-open this issue if you believe there truly is a bug with the code after taking into account that feedback. I'm also still working on a document that'll make it very clear how to reproduce the PRISM data (and all ERS data), but that'll take some time for us to do.