Fixing issue where GP model with masked data failed

[taylorbell57]

To fix the issue with masked data causing the GP model to fail, I've copied the code from the GP likelihood function which removed the masked data, and in the eval function I just add back masked zeros where there were masked values.

This PR pesolves #645

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 41.93548% with 18 lines in your changes are missing coverage. Please review.

Project coverage is 53.99%. Comparing base (75b0cbb) to head (6db706f).

Files	Patch %	Lines
...ightcurve_fitting/differentiable_models/GPModel.py	0.00%	16 Missing :warning:
src/eureka/S5_lightcurve_fitting/models/GPModel.py	86.66%	2 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #646 +/- ## ========================================== - Coverage 54.01% 53.99% -0.02% ========================================== Files 101 101 Lines 12692 12713 +21 ========================================== + Hits 6855 6864 +9 - Misses 5837 5849 +12 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[rluquer]

I've updated Eureka! to this branch. I do not see any improvement at all, except that the process does not break. However, the plots still do not show the GP model nor it seems that the GP parameters were properly sample. Maybe there's something I am not doing correctly. Happy to solve it offline.

[taylorbell57]

Can you confirm what Eureka version is installed with pip show eureka

[rluquer]

Name: eureka Version: 0.11.dev236+g413299ee Summary: A data reduction and analysis pipeline intended for time-series observations with JWST. Home-page: https://github.com/kevin218/Eureka Author: Eureka! pipeline developers Author-email: kbstevenson@gmail.com License: MIT License Location: /Users/rluque/miniconda3/envs/eureka/lib/python3.9/site-packages Requires: astraeus, astropy, astroquery, batman-package, bokeh, ccdproc, celerite2, corner, crds, dynesty, emcee, exotic-ld, george, h5py, lmfit, matplotlib, mc3, numpy, pandas, photutils, requests, scipy, setuptools-scm, stdatamodels, svo-filters, tqdm Required-by:

[taylorbell57]

So that's the same Eureka version that you'd previously had installed and not this newest version. You'll need to install the new version using something like pip install --upgrade 'eureka[jwst]@git+https://github.com/kevin218/Eureka.git@dev/tjb3'

[rluquer]

Not sure what to do. By doing that I get:

INFO: pip is looking at multiple versions of eureka to determine which version is compatible with other requirements. This could take a while. ERROR: Package 'eureka' requires a different Python: 3.9.7 not in '>=3.10'

[taylorbell57]

Oh right, the newer version of Eureka needs a higher version of Python (a choice forced on us by jwst requiring a higher Python version), so you'll have to start from scratch by building a new conda environment using Python 3.10

[rluquer]

Done that. Now, problem with george at installation. Nevertheless, I am trying to run it now.

Successfully built eureka astraeus Failed to build george ERROR: Could not build wheels for george, which is required to install pyproject.toml-based projects

[rluquer]

Installation is unsuccessful due to that error, cannot import eureka.

[taylorbell57]

Hmm, let me try to reproduce the george issue - what do you get from python --version?

[rluquer]

3.10.0

[taylorbell57]

Oh, are you running on Apple silicon (e.g. Apple M1 processor) or Intel?

[rluquer]

Apple M1 Pro, macOS Monterey v12.6.5

[taylorbell57]

Ah okay, so follow the installation instructions at https://eurekadocs.readthedocs.io/en/latest/installation.html#installing-with-a-conda-environment-yml-file, but replace the occurence of -b v0.10 with -b dev/tjb3

[rluquer]

Perfect, thank you! Sorry for the installation troubles, that is solved now.

I have tried now a S5 fit with GP and then getting the following error (happens with lsq, emcee and dynesty) regardless of the number of figures (because it happens at the starting point). It seems like it could be an easy fix.

========================= Starting lsq fit. Starting lnprob: 31140.476419251267

---

AttributeError Traceback (most recent call last) Cell In[17], line 1 ----> 1 s5_meta = eureka.S5_lightcurve_fitting.s5_fit.fitlc(eventlabel)

File ~/miniconda3/envs/eureka/lib/python3.10/site-packages/eureka/S5_lightcurve_fitting/s5_fit.py:482, in fitlc(eventlabel, ecf_path, s4_meta, input_meta) 476 # Normalize flux and uncertainties to avoid large 477 # flux values 478 flux, flux_err = util.normalize_spectrum( 479 meta, flux, flux_err, 480 scandir=getattr(lc, 'scandir', None)) --> 482 meta, params = fit_channel(meta, time_temp, flux, channel, 483 flux_err, eventlabel, params, 484 log, longparamlist, time_units, 485 paramtitles, chanrng, ld_coeffs, 486 xpos, ypos, xwidth, ywidth) 488 # Save results 489 log.writelog('Saving results', mute=(not meta.verbose))

File ~/miniconda3/envs/eureka/lib/python3.10/site-packages/eureka/S5_lightcurve_fitting/s5_fit.py:976, in fit_channel(meta, time, flux, chan, flux_err, eventlabel, params, log, longparamlist, time_units, paramtitles, chanrng, ldcoeffs, xpos, ypos, xwidth, ywidth, white) 974 log.writelog("Starting lsq fit.") 975 model.fitter = 'lsq' --> 976 lc_model.fit(model, meta, log, fitter='lsq') 977 log.writelog("Completed lsq fit.") 978 log.writelog("-------------------------")

File ~/miniconda3/envs/eureka/lib/python3.10/site-packages/eureka/S5_lightcurve_fitting/lightcurve.py:173, in LightCurve.fit(self, model, meta, log, fitter, kwargs) 170 raise ValueError("{} is not a valid fitter.".format(fitter)) 172 # Run the fit --> 173 fit_model = self.fitter_func(self, model, meta, log, kwargs) 175 # Store it 176 if fit_model is not None:

File ~/miniconda3/envs/eureka/lib/python3.10/site-packages/eureka/S5_lightcurve_fitting/fitters.py:81, in lsqfitter(lc, model, meta, log, calling_function, **kwargs) 79 # Plot starting point 80 if meta.isplots_S5 >= 1: ---> 81 plots.plot_fit(lc, model, meta, 82 fitter=calling_function+'StartingPoint') 83 # Plot GP starting point 84 if model.GP:

File ~/miniconda3/envs/eureka/lib/python3.10/site-packages/eureka/S5_lightcurve_fitting/plots_s5.py:139, in plot_fit(lc, model, meta, fitter, isTitle) 136 ax[2].set_ylabel('Residuals (ppm)', size=14) 137 ax[2].set_xlabel(str(lc.time_units), size=14) --> 139 fig.get_layout_engine().set(hspace=0, h_pad=0) 140 fig.align_ylabels(ax) 142 if lc.white:

AttributeError: 'NoneType' object has no attribute 'set'

[taylorbell57]

What do you get from pip show matplotlib? Seems like we probably need to specify a lower limit on the matplotlib version

[rluquer]

3.8.4

[taylorbell57]

Also did you run a line like eureka.lib.plots.set_rc(style='eureka') before running Stage 5 (e.g. in your run_eureka.py file)? You should be allowed to not do that, but it might explain the bug if you didn't run a line like that since the matplotlib layout engine may not have been initialized.

Matplotlib v3.8.4 should be fine

[rluquer]

I certainly did not! However, I have included that line and the code works now. The S5 fitting routine finished and all the plots were executed perfectly. Sorry for the trouble!

[taylorbell57]

Okay, that's good for me to know - glad this PR resolved your GP-related issue, and I'll open a separate issue to remind me about that matplotlib-related issue!