search

kevin218 / Eureka

Eureka! is a data reduction and analysis pipeline intended for time-series observations with JWST.
https://eurekadocs.readthedocs.io/
MIT License
56 stars 44 forks source link

[Bug]: Errors in Stage 5 using LD coefficients from a file #588

Closed Witchblade101 closed 7 months ago

Witchblade101 commented 7 months ago

FAQ check

Instrument

Light curve fitting (Stages 4-6)

What happened?

I tried to run Stage 5 using a file of pre-calculated limb-darkening coefficients. I wasn't sure in that case what to put for u1 and u2 in the .epf file, so I tried 2 different approaches: commenting out the u1, u2 entries or then setting them to dummy values.

I got errors in both cases. The first looks like it's just the wrong approach, but then the second looks like a bug.

Error traceback output

WARNING: There are multiple metadata save files in the folder: "/Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage4/" Using the metadata file: /Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage4/S4_2023-12-01_t1e_1_run1/ap6_bg7/S4_t1e_1_Meta_Save.dat and will consider aperture ranges listed there. If this metadata file is not a part of the run you intended, please provide a more precise folder for the metadata file. Finished loading parameters from /Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage4/S4_2023-12-01_t1e_1_run1/ap6_bg7//S4_t1e_1_ap6_bg7_LCData.h5

Starting Stage 5: Light Curve Fitting

Input directory: /Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage4/S4_2023-12-01_t1e_1_run1/ap6_bg7/ Output directory: /Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage5/S5_2023-12-01_t1e_1_run3/ap6_bg7/ Copying S5 control file Copying S5 parameter control file

Starting Fit of White-light Light Curve

Using the following limb-darkening values:

Starting lsq fit. 2023-12-01 21:04:23,054 - stpipe - WARNING - /Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/likelihood.py:134: RuntimeWarning: invalid value encountered in double_scalars lnprior_prob -= (0.5*(np.sum(((theta[i]-prior1[i])/prior2[i])**2

2023-12-01 21:04:23,055 - stpipe - WARNING - /Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/likelihood.py:135: RuntimeWarning: divide by zero encountered in log

Starting lnprob: -inf Traceback (most recent call last): File "/Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/run_eureka_1.py", line 28, in s5_meta = s5.fitlc(eventlabel, ecf_path=ecf_path) File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/s5_fit.py", line 290, in fitlc meta, params = fit_channel(meta, time, flux, 0, flux_err, File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/s5_fit.py", line 844, in fit_channel lc_model.fit(model, meta, log, fitter='lsq') File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/lightcurve.py", line 179, in fit fit_model = self.fitter_func(self, model, meta, log, *kwargs) File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/fitters.py", line 81, in lsqfitter plots.plot_fit(lc, model, meta, File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/plots_s5.py", line 52, in plot_fit model.physeval(interp=meta.interp) File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/models/Model.py", line 504, in physeval flux = component.eval(channel=channel, **kwargs) File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/models/BatmanModels.py", line 153, in eval m_transit = batman.TransitModel(bm_params, time, File "/Users/dlong/miniconda3/envs/eureka/lib/python3.9/site-packages/batman/transitmodel.py", line 78, in init raise Exception("Incorrect number of coefficients for " +params.limb_dark + " limb darkening; u should have the form:\n \ Exception: Incorrect number of coefficients for quadratic limb darkening; u should have the form: u = [] for uniform LD u = [u1] for linear LD u = [u1, u2] for quadratic, logarithmic, exponential, squareroot, and power2 LD u = [u1, u2, u3, u4] for nonlinear LD, or u = [u1, ..., un] for custom LD (eureka) minmei:Obs1 dlong$ (eureka) minmei:Obs1 dlong$ (eureka) minmei:Obs1 dlong$ (eureka) minmei:Obs1 dlong$ (eureka) minmei:Obs1 dlong$ (eureka) minmei:Obs1 dlong$ python run_eureka_1.py WARNING: There are multiple metadata save files in the folder: "/Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage4/" Using the metadata file: /Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage4/S4_2023-12-01_t1e_1_run1/ap6_bg7/S4_t1e_1_Meta_Save.dat and will consider aperture ranges listed there. If this metadata file is not a part of the run you intended, please provide a more precise folder for the metadata file. Finished loading parameters from /Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage4/S4_2023-12-01_t1e_1_run1/ap6_bg7//S4_t1e_1_ap6_bg7_LCData.h5

Starting Stage 5: Light Curve Fitting

Input directory: /Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage4/S4_2023-12-01_t1e_1_run1/ap6_bg7/ Output directory: /Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/Stage5/S5_2023-12-01_t1e_1_run4/ap6_bg7/ Copying S5 control file Copying S5 parameter control file

Starting Fit of White-light Light Curve

Using the following limb-darkening values: Traceback (most recent call last): File "/Users/dlong/DataAnalysis/JWST/eureka/t1e/Obs1/run_eureka_1.py", line 28, in s5_meta = s5.fitlc(eventlabel, ecf_path=ecf_path) File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/s5_fit.py", line 290, in fitlc meta, params = fit_channel(meta, time, flux, 0, flux_err, File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/s5_fit.py", line 589, in fit_channel t_transit = m.BatmanTransitModel(parameters=params, name='transit', File "/Users/dlong/Eureka/src/eureka/S5_lightcurve_fitting/models/BatmanModels.py", line 58, in init ld_val = ld_array[chan][param-1] UnboundLocalError: local variable 'ld_array' referenced before assignment

What operating system are you using?

Mac OS 13.6.1 Ventura

What version of Python are you running?

Python 3.9.7

What Python packages do you have installed?

packages in environment at /Users/dlong/miniconda3/envs/eureka:

#

Name Version Build Channel

aesara-theano-fallback 0.1.0 pypi_0 pypi alabaster 0.7.13 pypi_0 pypi anyio 4.0.0 pypi_0 pypi appnope 0.1.3 pypi_0 pypi argon2-cffi 23.1.0 pypi_0 pypi argon2-cffi-bindings 21.2.0 pypi_0 pypi arrow 1.3.0 pypi_0 pypi arviz 0.12.1 pypi_0 pypi asciitree 0.3.3 pypi_0 pypi asdf 2.15.0 pypi_0 pypi asdf-astropy 0.4.0 pypi_0 pypi asdf-coordinates-schemas 0.2.0 pypi_0 pypi asdf-standard 1.0.3 pypi_0 pypi asdf-transform-schemas 0.4.0 pypi_0 pypi asdf-unit-schemas 0.1.0 pypi_0 pypi asdf-wcs-schemas 0.2.0 pypi_0 pypi asteval 0.9.31 pypi_0 pypi astraeus 0.3 pypi_0 pypi astropy 5.3.4 pypi_0 pypi astropy-healpix 1.0.0 pypi_0 pypi astroquery 0.4.6 pypi_0 pypi astroscrappy 1.1.0 pypi_0 pypi asttokens 2.4.1 pypi_0 pypi async-lru 2.0.4 pypi_0 pypi attrs 23.1.0 pypi_0 pypi babel 2.13.1 pypi_0 pypi batman-package 2.4.9 pypi_0 pypi bayesicfitting 3.2.0 pypi_0 pypi beautifulsoup4 4.12.2 pypi_0 pypi bleach 6.1.0 pypi_0 pypi bokeh 2.4.3 pypi_0 pypi bottleneck 1.3.7 pypi_0 pypi ca-certificates 2023.7.22 h8857fd0_0 conda-forge cachetools 5.3.2 pypi_0 pypi ccdproc 2.4.1 pypi_0 pypi celerite 0.4.2 pypi_0 pypi certifi 2023.7.22 pypi_0 pypi cffi 1.16.0 pypi_0 pypi cftime 1.6.3 pypi_0 pypi charset-normalizer 3.3.2 pypi_0 pypi click 8.1.7 pypi_0 pypi cloudpickle 3.0.0 pypi_0 pypi comm 0.2.0 pypi_0 pypi contourpy 1.2.0 pypi_0 pypi corner 2.2.2 pypi_0 pypi coverage 7.3.2 pypi_0 pypi crds 11.17.9 pypi_0 pypi cycler 0.12.1 pypi_0 pypi dask 2023.10.1 pypi_0 pypi debugpy 1.8.0 pypi_0 pypi decorator 5.1.1 pypi_0 pypi defusedxml 0.7.1 pypi_0 pypi deprecat 2.1.1 pypi_0 pypi dill 0.3.7 pypi_0 pypi docutils 0.18.1 pypi_0 pypi drizzle 1.14.3 pypi_0 pypi dynesty 2.1.3 pypi_0 pypi emcee 3.1.4 pypi_0 pypi eureka 0.11.dev0+g696df7c8.d20231115 pypi_0 pypi exceptiongroup 1.1.3 pypi_0 pypi executing 2.0.1 pypi_0 pypi exoplanet 0.5.3 pypi_0 pypi exoplanet-core 0.1.2 pypi_0 pypi exotic-ld 3.0.0 pypi_0 pypi fasteners 0.19 pypi_0 pypi fastjsonschema 2.18.1 pypi_0 pypi fastprogress 1.0.3 pypi_0 pypi filelock 3.13.1 pypi_0 pypi flake8 6.1.0 pypi_0 pypi fonttools 4.44.0 pypi_0 pypi fqdn 1.5.1 pypi_0 pypi fsspec 2023.10.0 pypi_0 pypi future 0.18.3 pypi_0 pypi george 0.4.0 pypi_0 pypi gwcs 0.19.0 pypi_0 pypi h5netcdf 1.3.0 pypi_0 pypi h5py 3.10.0 pypi_0 pypi html5lib 1.1 pypi_0 pypi idna 3.4 pypi_0 pypi imageio 2.32.0 pypi_0 pypi imagesize 1.4.1 pypi_0 pypi importlib-metadata 6.8.0 pypi_0 pypi importlib-resources 6.1.1 pypi_0 pypi iniconfig 2.0.0 pypi_0 pypi ipykernel 6.26.0 pypi_0 pypi ipython 8.17.2 pypi_0 pypi ipywidgets 8.1.1 pypi_0 pypi isoduration 20.11.0 pypi_0 pypi jaraco-classes 3.3.0 pypi_0 pypi jedi 0.19.1 pypi_0 pypi jinja2 3.1.2 pypi_0 pypi jmespath 1.0.1 pypi_0 pypi json5 0.9.14 pypi_0 pypi jsonpointer 2.4 pypi_0 pypi jsonschema 4.17.3 pypi_0 pypi jsonschema-specifications 2023.7.1 pypi_0 pypi jupyter 1.0.0 pypi_0 pypi jupyter-client 8.6.0 pypi_0 pypi jupyter-console 6.6.3 pypi_0 pypi jupyter-core 5.5.0 pypi_0 pypi jupyter-events 0.9.0 pypi_0 pypi jupyter-lsp 2.2.0 pypi_0 pypi jupyter-server 2.10.0 pypi_0 pypi jupyter-server-terminals 0.4.4 pypi_0 pypi jupyterlab 4.0.8 pypi_0 pypi jupyterlab-pygments 0.2.2 pypi_0 pypi jupyterlab-server 2.25.1 pypi_0 pypi jupyterlab-widgets 3.0.9 pypi_0 pypi jwst 1.11.4 pypi_0 pypi keyring 24.2.0 pypi_0 pypi kiwisolver 1.4.5 pypi_0 pypi lazy-loader 0.3 pypi_0 pypi libffi 3.4.2 h0d85af4_5 conda-forge libsqlite 3.44.0 h92b6c6a_0 conda-forge libzlib 1.2.13 h8a1eda9_5 conda-forge llvm-openmp 17.0.4 hb6ac08f_0 conda-forge lmfit 1.2.2 pypi_0 pypi locket 1.0.0 pypi_0 pypi markdown-it-py 3.0.0 pypi_0 pypi markupsafe 2.1.3 pypi_0 pypi matplotlib 3.8.1 pypi_0 pypi matplotlib-inline 0.1.6 pypi_0 pypi mccabe 0.7.0 pypi_0 pypi mdit-py-plugins 0.4.0 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi mistune 3.0.2 pypi_0 pypi mkl 2022.2.1 h44ed08c_16952 conda-forge mkl-service 2.4.0 py39hcf0e074_0 conda-forge more-itertools 10.1.0 pypi_0 pypi myst-parser 2.0.0 pypi_0 pypi nbclient 0.9.0 pypi_0 pypi nbconvert 7.11.0 pypi_0 pypi nbformat 5.9.2 pypi_0 pypi nbsphinx 0.9.3 pypi_0 pypi ncurses 6.4 h93d8f39_2 conda-forge nest-asyncio 1.5.8 pypi_0 pypi netcdf4 1.6.5 pypi_0 pypi networkx 3.2.1 pypi_0 pypi notebook 7.0.6 pypi_0 pypi notebook-shim 0.2.3 pypi_0 pypi numcodecs 0.12.1 pypi_0 pypi numpy 1.21.6 pypi_0 pypi numpydoc 1.6.0 pypi_0 pypi opencv-python 4.7.0.72 pypi_0 pypi opencv-python-headless 4.7.0.72 pypi_0 pypi openssl 3.1.4 hd75f5a5_0 conda-forge overrides 7.4.0 pypi_0 pypi packaging 23.2 pypi_0 pypi pandas 2.0.3 pypi_0 pypi pandocfilters 1.5.0 pypi_0 pypi parsley 1.3 pypi_0 pypi parso 0.8.3 pypi_0 pypi partd 1.4.1 pypi_0 pypi patsy 0.5.3 pypi_0 pypi pexpect 4.8.0 pypi_0 pypi photutils 1.8.0 pypi_0 pypi pillow 10.0.1 pypi_0 pypi pip 23.3.1 pyhd8ed1ab_0 conda-forge platformdirs 3.11.0 pypi_0 pypi pluggy 1.3.0 pypi_0 pypi poppy 1.1.1 pypi_0 pypi prometheus-client 0.18.0 pypi_0 pypi prompt-toolkit 3.0.39 pypi_0 pypi psutil 5.9.6 pypi_0 pypi ptyprocess 0.7.0 pypi_0 pypi pure-eval 0.2.2 pypi_0 pypi pycodestyle 2.11.1 pypi_0 pypi pycparser 2.21 pypi_0 pypi pyerfa 2.0.1.1 pypi_0 pypi pyflakes 3.1.0 pypi_0 pypi pygments 2.16.1 pypi_0 pypi pymc3 3.11.5 pypi_0 pypi pymc3-ext 0.1.1 pypi_0 pypi pyparsing 3.1.1 pypi_0 pypi pyrsistent 0.20.0 pypi_0 pypi pysynphot 2.0.0 pypi_0 pypi pytest 7.4.3 pypi_0 pypi pytest-cov 4.1.0 pypi_0 pypi pytest-doctestplus 1.0.0 pypi_0 pypi python 3.9.7 h38b4d05_3_cpython conda-forge python-dateutil 2.8.2 pypi_0 pypi python-json-logger 2.0.7 pypi_0 pypi python_abi 3.9 4_cp39 conda-forge pytz 2023.3.post1 pypi_0 pypi pyvo 1.4.2 pypi_0 pypi pywavelets 1.4.1 pypi_0 pypi pyyaml 6.0.1 pypi_0 pypi pyzmq 25.1.1 pypi_0 pypi qtconsole 5.5.0 pypi_0 pypi qtpy 2.4.1 pypi_0 pypi readline 8.2 h9e318b2_1 conda-forge referencing 0.30.2 pypi_0 pypi reproject 0.12.0 pypi_0 pypi requests 2.31.0 pypi_0 pypi rfc3339-validator 0.1.4 pypi_0 pypi rfc3986-validator 0.1.1 pypi_0 pypi rpds-py 0.12.0 pypi_0 pypi scikit-image 0.19.3 pypi_0 pypi scipy 1.7.3 pypi_0 pypi semantic-version 2.10.0 pypi_0 pypi semver 3.0.2 pypi_0 pypi send2trash 1.8.2 pypi_0 pypi setuptools 68.2.2 pyhd8ed1ab_0 conda-forge setuptools-scm 8.0.4 pypi_0 pypi six 1.16.0 pyh6c4a22f_0 conda-forge sniffio 1.3.0 pypi_0 pypi snowballstemmer 2.2.0 pypi_0 pypi soupsieve 2.5 pypi_0 pypi spherical-geometry 1.3.1 pypi_0 pypi sphinx 7.2.6 pypi_0 pypi sphinx-automodapi 0.16.0 pypi_0 pypi sphinx-rtd-theme 1.3.0 pypi_0 pypi sphinxcontrib-applehelp 1.0.7 pypi_0 pypi sphinxcontrib-devhelp 1.0.5 pypi_0 pypi sphinxcontrib-htmlhelp 2.0.4 pypi_0 pypi sphinxcontrib-jquery 4.1 pypi_0 pypi sphinxcontrib-jsmath 1.0.1 pypi_0 pypi sphinxcontrib-qthelp 1.0.6 pypi_0 pypi sphinxcontrib-serializinghtml 1.1.9 pypi_0 pypi sqlite 3.44.0 h7461747_0 conda-forge stack-data 0.6.3 pypi_0 pypi starry 1.2.0 pypi_0 pypi stcal 1.4.4 pypi_0 pypi stdatamodels 1.7.2 pypi_0 pypi stpipe 0.5.1 pypi_0 pypi stsci-image 2.3.5 pypi_0 pypi stsci-imagestats 1.6.3 pypi_0 pypi stsci-stimage 0.2.6 pypi_0 pypi svo-filters 0.4.4 pypi_0 pypi tabulate 0.9.0 pypi_0 pypi tbb 2021.10.0 h1c7c39f_2 conda-forge terminado 0.17.1 pypi_0 pypi theano-pymc 1.1.2 pypi_0 pypi tifffile 2023.9.26 pypi_0 pypi tinycss2 1.2.1 pypi_0 pypi tk 8.6.13 h1abcd95_1 conda-forge tomli 2.0.1 pypi_0 pypi toolz 0.12.0 pypi_0 pypi tornado 6.3.3 pypi_0 pypi tqdm 4.66.1 pypi_0 pypi traitlets 5.13.0 pypi_0 pypi tweakwcs 0.8.3 pypi_0 pypi types-python-dateutil 2.8.19.14 pypi_0 pypi typing-extensions 4.8.0 pypi_0 pypi tzdata 2023.3 pypi_0 pypi uncertainties 3.1.7 pypi_0 pypi uri-template 1.3.0 pypi_0 pypi urllib3 2.0.7 pypi_0 pypi wcwidth 0.2.9 pypi_0 pypi webcolors 1.13 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi websocket-client 1.6.4 pypi_0 pypi wheel 0.41.3 pyhd8ed1ab_0 conda-forge widgetsnbextension 4.0.9 pypi_0 pypi wiimatch 0.3.1 pypi_0 pypi wrapt 1.16.0 pypi_0 pypi xarray 2023.9.0 pypi_0 pypi xarray-einstats 0.6.0 pypi_0 pypi xz 5.2.6 h775f41a_0 conda-forge zarr 2.16.1 pypi_0 pypi zipp 3.17.0 pypi_0 pypi zlib 1.2.13 h8a1eda9_5 conda-forge

Code of Conduct

kevin218 commented 7 months ago

I think the problem is that the file is in the wrong format. Here is the file we're trying to use: prism-ldcoeff-u1-u2-TRAPPIST1.txt but Eureka! is looking for one row per wavelength bin and one column per LD parameter (i.e., no wavelength column). We either need to implement new code in Stage 4 that reads in this LD file or we need to ask Néstor to create a new LD file with agreed upon wavelength bins.

Witchblade101 commented 7 months ago

In this case I think the second option makes the most sense: agree upon a wavelength range and number of bins, then generate a fixed LD file for, as an example, 100 bins between 0.5 and 5.6 microns.

Witchblade101 commented 7 months ago

I tried a test using an edited LD file. It only had 411 rows (wavelengths 0.59715 to 5.30400 microns) and no wavelength column. I ran stage 4 with 411 bins with a min wavelength of 0.6 and max of 5.3. Specifying the new file, Stage 5 still fails with the "UnboundLocalError: local variable 'ld_array' referenced before assignment" error

kevin218 commented 7 months ago

@Witchblade101 Can you send me your custom LD file so that I can try to find a fix? Sending the Stage 4 LCData.h5 file and Stage 5 ECF/EPF would also help.

Witchblade101 commented 7 months ago

prism-ldcoeff-u1-u2-TRAPPIST1-test.txt

kevin218 commented 7 months ago

I've submitted a PR (#589) that addresses this issue. Once it is merged into main, you'll need to update your version of Eureka!.

kevin218 commented 7 months ago

Completed via https://github.com/kevin218/Eureka/pull/589.