[Bug]: Plotting problem in S5 when using GPs

rluquer commented 5 months ago

FAQ check

[X] Yes, I checked the FAQ and my question has not been addressed.

Instrument

Light curve fitting (Stages 4-6)

What happened?

When using GPs (celerite, Matern32 kernel) on a S5 fit, there is a problem with the plotting that impedes the function s5_meta = eureka.S5_lightcurve_fitting.s5_fit.fitlc(eventlabel) to finish, thus not enabling S6 to run properly. The actual fit works and finishes (I've tried lsq and dynesty, both works), the problem seems to be with plotting when isplots_S5 >= 3.

Error traceback output

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[17], line 1
----> 1 s5_meta = eureka.S5_lightcurve_fitting.s5_fit.fitlc(eventlabel)

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/s5_fit.py:480, in fitlc(eventlabel, ecf_path, s4_meta, input_meta)
    475 # Normalize flux and uncertainties to avoid large
    476 # flux values
    477 flux, flux_err = util.normalize_spectrum(meta, flux,
    478                                          flux_err)
--> 480 meta, params = fit_channel(meta, time_temp, flux, channel,
    481                            flux_err, eventlabel, params,
    482                            log, longparamlist, time_units,
    483                            paramtitles, chanrng, ld_coeffs,
    484                            xpos, ypos, xwidth, ywidth)
    486 # Save results
    487 log.writelog('Saving results', mute=(not meta.verbose))

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/s5_fit.py:986, in fit_channel(meta, time, flux, chan, flux_err, eventlabel, params, log, longparamlist, time_units, paramtitles, chanrng, ldcoeffs, xpos, ypos, xwidth, ywidth, white)
    984 log.writelog("Starting dynesty fit.")
    985 model.fitter = 'dynesty'
--> 986 lc_model.fit(model, meta, log, fitter='dynesty')
    987 log.writelog("Completed dynesty fit.")
    988 log.writelog("-------------------------")

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/lightcurve.py:173, in LightCurve.fit(self, model, meta, log, fitter, **kwargs)
    170     raise ValueError("{} is not a valid fitter.".format(fitter))
    172 # Run the fit
--> 173 fit_model = self.fitter_func(self, model, meta, log, **kwargs)
    175 # Store it
    176 if fit_model is not None:

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/fitters.py:918, in dynestyfitter(lc, model, meta, log, **kwargs)
    916 # Plot residuals distribution
    917 if meta.isplots_S5 >= 3:
--> 918     plots.plot_res_distr(lc, model, meta, fitter='dynesty')
    920 # plot using corner.py
    921 if meta.isplots_S5 >= 5:

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/plots_s5.py:683, in plot_res_distr(lc, model, meta, fitter)
    680 # Mask out any infinities or nans
    681 hist_vals = np.ma.masked_invalid(hist_vals)
--> 683 n, bins, patches = plt.hist(hist_vals, alpha=0.5, color='b',
    684                             edgecolor='b', lw=1)
    685 x = np.linspace(-4., 4., 200)
    686 px = stats.norm.pdf(x, loc=0, scale=1)

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/matplotlib/pyplot.py:3236, in hist(x, bins, range, density, weights, cumulative, bottom, histtype, align, orientation, rwidth, log, color, label, stacked, data, **kwargs)
   3211 @_copy_docstring_and_deprecators(Axes.hist)
   3212 def hist(
   3213     x: ArrayLike | Sequence[ArrayLike],
   (...)
   3234     BarContainer | Polygon | list[BarContainer | Polygon],
   3235 ]:
-> 3236     return gca().hist(
   3237         x,
   3238         bins=bins,
   3239         range=range,
   3240         density=density,
   3241         weights=weights,
   3242         cumulative=cumulative,
   3243         bottom=bottom,
   3244         histtype=histtype,
   3245         align=align,
   3246         orientation=orientation,
   3247         rwidth=rwidth,
   3248         log=log,
   3249         color=color,
   3250         label=label,
   3251         stacked=stacked,
   3252         **({"data": data} if data is not None else {}),
   3253         **kwargs,
   3254     )

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/matplotlib/__init__.py:1465, in _preprocess_data.<locals>.inner(ax, data, *args, **kwargs)
   1462 @functools.wraps(func)
   1463 def inner(ax, *args, data=None, **kwargs):
   1464     if data is None:
-> 1465         return func(ax, *map(sanitize_sequence, args), **kwargs)
   1467     bound = new_sig.bind(ax, *args, **kwargs)
   1468     auto_label = (bound.arguments.get(label_namer)
   1469                   or bound.kwargs.get(label_namer))

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/matplotlib/axes/_axes.py:6862, in Axes.hist(self, x, bins, range, density, weights, cumulative, bottom, histtype, align, orientation, rwidth, log, color, label, stacked, **kwargs)
   6858 # Loop through datasets
   6859 for i in range(nx):
   6860     # this will automatically overwrite bins,
   6861     # so that each histogram uses the same bins
-> 6862     m, bins = np.histogram(x[i], bins, weights=w[i], **hist_kwargs)
   6863     tops.append(m)
   6864 tops = np.array(tops, float)  # causes problems later if it's an int

File <__array_function__ internals>:180, in histogram(*args, **kwargs)

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/numpy/lib/histograms.py:793, in histogram(a, bins, range, normed, weights, density)
    681 r"""
    682 Compute the histogram of a dataset.
    683 
   (...)
    789 
    790 """
    791 a, weights = _ravel_and_check_weights(a, weights)
--> 793 bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)
    795 # Histogram is an integer or a float array depending on the weights.
    796 if weights is None:

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/numpy/lib/histograms.py:426, in _get_bin_edges(a, bins, range, weights)
    423     if n_equal_bins < 1:
    424         raise ValueError('`bins` must be positive, when an integer')
--> 426     first_edge, last_edge = _get_outer_edges(a, range)
    428 elif np.ndim(bins) == 1:
    429     bin_edges = np.asarray(bins)

File ~/miniconda3/envs/eureka/lib/python3.9/site-packages/numpy/lib/histograms.py:323, in _get_outer_edges(a, range)
    321     first_edge, last_edge = a.min(), a.max()
    322     if not (np.isfinite(first_edge) and np.isfinite(last_edge)):
--> 323         raise ValueError(
    324             "autodetected range of [{}, {}] is not finite".format(first_edge, last_edge))
    326 # expand empty range to avoid divide by zero
    327 if first_edge == last_edge:

ValueError: autodetected range of [nan, nan] is not finite

What operating system are you using?

MacOS Monterey (12.6.5)

What version of Python are you running?

Python 3.9.7

What Python packages do you have installed?

# packages in environment at /Users/rluque/miniconda3/envs/eureka:
#
# Name                    Version                   Build  Channel
affine                    2.4.0              pyhd8ed1ab_0    conda-forge
alabaster                 0.7.16             pyhd8ed1ab_0    conda-forge
anyio                     4.2.0              pyhd8ed1ab_0    conda-forge
aom                       3.8.1                h078ce10_0    conda-forge
appnope                   0.1.4              pyhd8ed1ab_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0           py39h0f82c59_4    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
arviz                     0.17.0             pyhd8ed1ab_0    conda-forge
asciitree                 0.3.3                      py_2    conda-forge
asdf                      3.0.1              pyhd8ed1ab_0    conda-forge
asdf-astropy              0.5.0              pyhd8ed1ab_0    conda-forge
asdf-coordinates-schemas  0.2.0              pyhd8ed1ab_1    conda-forge
asdf-standard             1.0.3              pyhd8ed1ab_0    conda-forge
asdf-transform-schemas    0.4.0              pyhd8ed1ab_0    conda-forge
asdf-unit-schemas         0.1.0              pyhd8ed1ab_0    conda-forge
asdf-wcs-schemas          0.3.0              pyhd8ed1ab_0    conda-forge
asteval                   0.9.31             pyhd8ed1ab_0    conda-forge
astraeus                  0.3                      pypi_0    pypi
astropy                   6.0.0            py39h373d45f_0    conda-forge
astropy-healpix           1.0.2            py39h373d45f_1    conda-forge
astropy-iers-data         0.2024.2.12.0.31.0    pyhd8ed1ab_0    conda-forge
astroquery                0.4.6                    pypi_0    pypi
astroscrappy              1.1.0            py39h4d8bf0d_2    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0             pyh71513ae_0    conda-forge
aws-c-auth                0.7.15               h8117f06_0    conda-forge
aws-c-cal                 0.6.9                h4fd42c2_3    conda-forge
aws-c-common              0.9.12               h93a5062_0    conda-forge
aws-c-compression         0.2.17               h4fd42c2_8    conda-forge
aws-c-event-stream        0.4.1                hf6cc7c5_5    conda-forge
aws-c-http                0.8.0                hf1748bb_5    conda-forge
aws-c-io                  0.14.3               h8daa835_1    conda-forge
aws-c-mqtt                0.10.1               h7f0f801_3    conda-forge
aws-c-s3                  0.5.0                hbb97ff1_2    conda-forge
aws-c-sdkutils            0.1.14               h4fd42c2_0    conda-forge
aws-checksums             0.1.17               h4fd42c2_7    conda-forge
aws-crt-cpp               0.26.1               h9b04f48_9    conda-forge
aws-sdk-cpp               1.11.242             h26e3666_0    conda-forge
azure-core-cpp            1.10.3               he231e37_1    conda-forge
azure-storage-blobs-cpp   12.10.0              h6aa02a4_0    conda-forge
azure-storage-common-cpp  12.5.0               h607ffeb_2    conda-forge
babel                     2.14.0             pyhd8ed1ab_0    conda-forge
batman-package            2.4.9            py39h4d8bf0d_1    conda-forge
bayesicfitting            3.2.0                    pypi_0    pypi
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
blosc                     1.21.5               hc338f07_0    conda-forge
bokeh                     2.4.3              pyhd8ed1ab_3    conda-forge
bottleneck                1.3.7            py39hf4a74a7_1    conda-forge
brotli                    1.1.0                hb547adb_1    conda-forge
brotli-bin                1.1.0                hb547adb_1    conda-forge
brotli-python             1.1.0            py39hb198ff7_1    conda-forge
brunsli                   0.1                  h9f76cd9_0    conda-forge
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.26.0               h93a5062_0    conda-forge
c-blosc2                  2.13.2               ha57e6be_0    conda-forge
ca-certificates           2024.2.2             hf0a4a13_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cairo                     1.18.0               hd1e100b_0    conda-forge
ccdproc                   2.4.1              pyhd8ed1ab_0    conda-forge
celerite                  0.4.2            py39h425d09f_4    conda-forge
celerite2                 0.3.0                    pypi_0    pypi
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cffi                      1.16.0           py39he153c15_0    conda-forge
cfitsio                   4.3.1                h808cd33_0    conda-forge
cftime                    1.6.3                    pypi_0    pypi
charls                    2.4.2                h13dd4ca_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.1              pyhd8ed1ab_0    conda-forge
contourpy                 1.2.0            py39he9de807_0    conda-forge
corner                    2.2.2              pyhd8ed1ab_0    conda-forge
coverage                  7.4.1            py39h17cfd9d_0    conda-forge
crds                      11.17.16                 pypi_0    pypi
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
cytoolz                   0.12.3           py39h17cfd9d_0    conda-forge
dask                      2024.2.0           pyhd8ed1ab_0    conda-forge
dask-core                 2024.2.0           pyhd8ed1ab_0    conda-forge
dav1d                     1.2.1                hb547adb_0    conda-forge
debugpy                   1.8.1            py39hf3050f2_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
distributed               2024.2.0           pyhd8ed1ab_0    conda-forge
docutils                  0.20.1           py39h2804cbe_3    conda-forge
drizzle                   1.15.0                   pypi_0    pypi
dynesty                   2.1.2              pyhd8ed1ab_0    conda-forge
emcee                     3.1.4              pyhd8ed1ab_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
eureka                    0.11.dev236+g413299ee          pypi_0    pypi
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
exotic-ld                 3.0.0                    pypi_0    pypi
expat                     2.5.0                hb7217d7_1    conda-forge
fasteners                 0.17.3             pyhd8ed1ab_0    conda-forge
filelock                  3.13.1                   pypi_0    pypi
flake8                    7.0.0              pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_1    conda-forge
fontconfig                2.14.2               h82840c6_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.49.0           py39h17cfd9d_0    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freetype                  2.12.1               hadb7bae_2    conda-forge
freexl                    2.0.0                hfbad9fb_0    conda-forge
fsspec                    2024.2.0           pyhca7485f_0    conda-forge
future                    0.18.3             pyhd8ed1ab_0    conda-forge
george                    0.4.0            py39hf8cecc8_6    conda-forge
geos                      3.12.1               h965bd2d_0    conda-forge
geotiff                   1.7.1               h7bcba05_15    conda-forge
gettext                   0.21.1               h0186832_0    conda-forge
gflags                    2.2.2             hc88da5d_1004    conda-forge
giflib                    5.2.1                h1a8c8d9_3    conda-forge
glog                      0.6.0                h6da1cb0_0    conda-forge
gwcs                      0.20.0             pyhd8ed1ab_0    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
h5netcdf                  1.3.0              pyhd8ed1ab_0    conda-forge
h5py                      3.10.0          nompi_py39hd62a535_101    conda-forge
hdf4                      4.2.15               h2ee6834_7    conda-forge
hdf5                      1.14.3          nompi_h5bb55e9_100    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
html5lib                  1.1                      pypi_0    pypi
httpcore                  1.0.3              pyhd8ed1ab_0    conda-forge
httpx                     0.26.0             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
image-registration        0.2.8.dev12+gf95d959          pypi_0    pypi
imagecodecs               2024.1.1         py39h60fbabe_0    conda-forge
imageio                   2.34.0             pyh4b66e23_0    conda-forge
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        7.0.1              pyha770c72_0    conda-forge
importlib-resources       6.1.1              pyhd8ed1ab_0    conda-forge
importlib_metadata        7.0.1                hd8ed1ab_0    conda-forge
importlib_resources       6.1.1              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
ipykernel                 6.29.2             pyh3cd1d5f_0    conda-forge
ipython                   8.18.1             pyh707e725_3    conda-forge
ipywidgets                8.1.2              pyhd8ed1ab_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
jaraco-classes            3.3.1                    pypi_0    pypi
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.3              pyhd8ed1ab_0    conda-forge
jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
json-c                    0.17                 h40ed0f5_0    conda-forge
json5                     0.9.14             pyhd8ed1ab_0    conda-forge
jsonpointer               2.4              py39h2804cbe_3    conda-forge
jsonschema                4.21.1             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.21.1             pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0             pyhd8ed1ab_10    conda-forge
jupyter-lsp               2.2.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.0              pyhd8ed1ab_0    conda-forge
jupyter_console           6.6.3              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.1            py39h2804cbe_0    conda-forge
jupyter_events            0.9.0              pyhd8ed1ab_0    conda-forge
jupyter_server            2.12.5             pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.5.2              pyhd8ed1ab_0    conda-forge
jupyterlab                4.1.1              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.3.0              pyhd8ed1ab_1    conda-forge
jupyterlab_server         2.25.3             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.10             pyhd8ed1ab_0    conda-forge
jwst                      1.12.2                   pypi_0    pypi
jxrlib                    1.1                  h93a5062_3    conda-forge
kealib                    1.5.3                h210d843_0    conda-forge
keyring                   24.3.0                   pypi_0    pypi
kiwisolver                1.4.5            py39hbd775c9_1    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
lazy_loader               0.3                pyhd8ed1ab_0    conda-forge
lcms2                     2.16                 ha0e7c42_0    conda-forge
lerc                      4.0.0                h9a09cb3_0    conda-forge
libabseil                 20230802.1      cxx17_h13dd4ca_0    conda-forge
libaec                    1.1.2                h13dd4ca_1    conda-forge
libarchive                3.7.2                hcacb583_1    conda-forge
libarrow                  15.0.0           h906e67b_2_cpu    conda-forge
libarrow-acero            15.0.0           h13dd4ca_2_cpu    conda-forge
libarrow-dataset          15.0.0           h13dd4ca_2_cpu    conda-forge
libarrow-flight           15.0.0           h7c660a6_2_cpu    conda-forge
libarrow-flight-sql       15.0.0           h6dff610_2_cpu    conda-forge
libarrow-gandiva          15.0.0           hf757142_2_cpu    conda-forge
libarrow-substrait        15.0.0           h3c8a37a_2_cpu    conda-forge
libavif16                 1.0.4                h76840f6_0    conda-forge
libblas                   3.9.0           21_osxarm64_openblas    conda-forge
libboost-headers          1.84.0               hce30654_1    conda-forge
libbrotlicommon           1.1.0                hb547adb_1    conda-forge
libbrotlidec              1.1.0                hb547adb_1    conda-forge
libbrotlienc              1.1.0                hb547adb_1    conda-forge
libcblas                  3.9.0           21_osxarm64_openblas    conda-forge
libcrc32c                 1.1.2                hbdafb3b_0    conda-forge
libcurl                   8.5.0                h2d989ff_0    conda-forge
libcxx                    16.0.6               h4653b0c_0    conda-forge
libdeflate                1.19                 hb547adb_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libevent                  2.1.12               h2757513_1    conda-forge
libexpat                  2.5.0                hb7217d7_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgdal                   3.8.3                ha86f356_3    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_3    conda-forge
libgfortran5              13.2.0               hf226fd6_3    conda-forge
libglib                   2.78.3               hb438215_0    conda-forge
libgoogle-cloud           2.12.0               h49bbb43_5    conda-forge
libgrpc                   1.60.1               hfc68871_0    conda-forge
libiconv                  1.17                 h0d3ecfb_2    conda-forge
libjpeg-turbo             3.0.0                hb547adb_1    conda-forge
libkml                    1.3.0             h1eb4d9f_1018    conda-forge
liblapack                 3.9.0           21_osxarm64_openblas    conda-forge
libllvm15                 15.0.7               h2621b3d_4    conda-forge
libnetcdf                 4.9.2           nompi_h291a7c2_113    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libopenblas               0.3.26          openmp_h6c19121_0    conda-forge
libparquet                15.0.0           hf6ce1d5_2_cpu    conda-forge
libpng                    1.6.42               h091b4b1_0    conda-forge
libpq                     16.2                 h0f8b458_0    conda-forge
libprotobuf               4.25.1               h810fc01_1    conda-forge
libre2-11                 2023.06.02           h1753957_0    conda-forge
librttopo                 1.1.0               hc8f776e_15    conda-forge
libsodium                 1.0.18               h27ca646_1    conda-forge
libspatialite             5.1.0                h69abc6b_4    conda-forge
libsqlite                 3.45.1               h091b4b1_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libthrift                 0.19.0               h026a170_1    conda-forge
libtiff                   4.6.0                ha8a6c65_2    conda-forge
libutf8proc               2.8.0                h1a8c8d9_0    conda-forge
libwebp-base              1.3.2                hb547adb_0    conda-forge
libxcb                    1.15                 hf346824_0    conda-forge
libxml2                   2.12.5               h0d0cfa8_0    conda-forge
libzip                    1.10.1               ha0bc3c6_3    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
libzopfli                 1.0.3                h9f76cd9_0    conda-forge
llvm-openmp               17.0.6               hcd81f8e_0    conda-forge
lmfit                     1.2.2              pyhd8ed1ab_1    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
lz4                       4.3.3            py39hf99b9d6_0    conda-forge
lz4-c                     1.9.4                hb7217d7_0    conda-forge
lzo                       2.10              h642e427_1000    conda-forge
markdown-it-py            3.0.0                    pypi_0    pypi
markupsafe                2.1.5            py39h17cfd9d_0    conda-forge
matplotlib                3.8.3            py39hdf13c20_0    conda-forge
matplotlib-base           3.8.3            py39hbab7938_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mc3                       3.1.3                    pypi_0    pypi
mccabe                    0.7.0              pyhd8ed1ab_0    conda-forge
mdit-py-plugins           0.4.0                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
minizip                   4.0.4                hc35e051_0    conda-forge
mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
more-itertools            10.2.0                   pypi_0    pypi
msgpack-python            1.0.7            py39he9de807_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
myst-parser               2.0.0                    pypi_0    pypi
nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
nbconvert                 7.16.0             pyhd8ed1ab_0    conda-forge
nbconvert-core            7.16.0             pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.16.0             pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
nbsphinx                  0.9.3              pyhd8ed1ab_0    conda-forge
ncurses                   6.4                  h463b476_2    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
netcdf4                   1.6.5                    pypi_0    pypi
networkx                  3.2.1              pyhd8ed1ab_0    conda-forge
notebook                  7.1.0              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.4              pyhd8ed1ab_0    conda-forge
nspr                      4.35                 hb7217d7_0    conda-forge
nss                       3.98                 h5ce2875_0    conda-forge
numcodecs                 0.12.1           py39h4ce5507_0    conda-forge
numpy                     1.23.0           py39h7df2422_0    conda-forge
numpydoc                  1.6.0              pyhd8ed1ab_0    conda-forge
opencv-python-headless    4.9.0.80                 pypi_0    pypi
openjpeg                  2.5.0                h4c1507b_3    conda-forge
openssl                   3.2.1                h0d3ecfb_0    conda-forge
orc                       1.9.2                hb41d57e_1    conda-forge
overrides                 7.7.0              pyhd8ed1ab_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.2.0            py39h47e51b9_0    conda-forge
pandoc                    3.1.12               hce30654_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parsley                   1.3                      pypi_0    pypi
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.4.1              pyhd8ed1ab_0    conda-forge
pcre2                     10.42                h26f9a81_0    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
photutils                 1.11.0           py39h373d45f_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.2.0           py39h755f0b7_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pixman                    0.43.2               hebf3989_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.4.0              pyhd8ed1ab_0    conda-forge
poppler                   24.02.0              h896e6cb_0    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
poppy                     1.1.1                    pypi_0    pypi
postgresql                16.2                 h1d0603d_0    conda-forge
proj                      9.3.1                h93d94ba_0    conda-forge
prometheus_client         0.20.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
prompt_toolkit            3.0.42               hd8ed1ab_0    conda-forge
psutil                    5.9.8            py39h17cfd9d_0    conda-forge
pthread-stubs             0.4               h27ca646_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   15.0.0          py39hb25a2a0_2_cpu    conda-forge
pyarrow-hotfix            0.6                pyhd8ed1ab_0    conda-forge
pycodestyle               2.11.1             pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyerfa                    2.0.1.1          py39h373d45f_0    conda-forge
pyflakes                  3.2.0              pyhd8ed1ab_0    conda-forge
pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
pyobjc-core               10.1             py39hb167abd_0    conda-forge
pyobjc-framework-cocoa    10.1             py39hb167abd_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pysynphot                 2.0.0                    pypi_0    pypi
pytest                    8.0.0              pyhd8ed1ab_0    conda-forge
pytest-cov                4.1.0              pyhd8ed1ab_0    conda-forge
pytest-doctestplus        1.1.0              pyhd8ed1ab_0    conda-forge
python                    3.9.7           hc0da0df_3_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.19.1             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python_abi                3.9                      4_cp39    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyvo                      1.5.1                    pypi_0    pypi
pywavelets                1.4.1            py39hf4a74a7_1    conda-forge
pyyaml                    6.0.1            py39h0f82c59_1    conda-forge
pyzmq                     25.1.2           py39he1e2164_0    conda-forge
qtconsole-base            5.5.1              pyha770c72_0    conda-forge
qtpy                      2.4.1              pyhd8ed1ab_0    conda-forge
rasterio                  1.3.9            py39hf398981_2    conda-forge
rav1e                     0.6.6                h69fbcac_2    conda-forge
re2                       2023.06.02           h6135d0a_0    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.33.0             pyhd8ed1ab_0    conda-forge
reproject                 0.13.0           py39h373d45f_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rpds-py                   0.18.0           py39h9a407ce_0    conda-forge
scikit-image              0.22.0           py39hf8cecc8_2    conda-forge
scikit-learn              1.4.1.post1      py39h6dd658b_0    conda-forge
scipy                     1.9.3                    pypi_0    pypi
semantic_version          2.10.0             pyhd8ed1ab_0    conda-forge
send2trash                1.8.2              pyhd1c38e8_0    conda-forge
setuptools                69.0.3             pyhd8ed1ab_0    conda-forge
setuptools-scm            8.0.4              pyhd8ed1ab_0    conda-forge
shapely                   2.0.2            py39ha70ab96_1    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               h17c5cce_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
snuggs                    1.4.7                      py_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
spherical-geometry        1.3.1                    pypi_0    pypi
sphinx                    7.2.6              pyhd8ed1ab_0    conda-forge
sphinx-automodapi         0.16.0             pyh6ff6d48_0    conda-forge
sphinx-rtd-theme          2.0.0                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.8              pyhd8ed1ab_0    conda-forge
sphinxcontrib-devhelp     1.0.6              pyhd8ed1ab_0    conda-forge
sphinxcontrib-htmlhelp    2.0.5              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jquery      4.1                      pypi_0    pypi
sphinxcontrib-jsmath      1.0.1              pyhd8ed1ab_0    conda-forge
sphinxcontrib-qthelp      1.0.7              pyhd8ed1ab_0    conda-forge
sphinxcontrib-serializinghtml 1.1.10             pyhd8ed1ab_0    conda-forge
sqlite                    3.45.1               hf2abe2d_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
stcal                     1.4.4                    pypi_0    pypi
stdatamodels              1.8.4                    pypi_0    pypi
stpipe                    0.5.1                    pypi_0    pypi
stsci-image               2.3.5                    pypi_0    pypi
stsci-imagestats          1.8.0                    pypi_0    pypi
stsci-stimage             0.2.6                    pypi_0    pypi
svo-filters               0.4.4                    pypi_0    pypi
svt-av1                   1.8.0                h463b476_0    conda-forge
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tblib                     3.0.0              pyhd8ed1ab_0    conda-forge
terminado                 0.18.0             pyh31c8845_0    conda-forge
threadpoolctl             3.3.0              pyhc1e730c_0    conda-forge
tifffile                  2024.2.12          pyhd8ed1ab_0    conda-forge
tiledb                    2.20.0               h49d9ff7_0    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3            py39h0f82c59_1    conda-forge
tqdm                      4.66.2             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.1             pyhd8ed1ab_0    conda-forge
tweakwcs                  0.8.6                    pypi_0    pypi
types-python-dateutil     2.8.19.20240106    pyhd8ed1ab_0    conda-forge
typing-extensions         4.9.0                hd8ed1ab_0    conda-forge
typing_extensions         4.9.0              pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzcode                    2024a                h93a5062_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
uncertainties             3.1.7              pyhd8ed1ab_0    conda-forge
unicodedata2              15.1.0           py39h0f82c59_0    conda-forge
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
uriparser                 0.9.7                hb7217d7_1    conda-forge
urllib3                   2.2.0              pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
webcolors                 1.13               pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.7.0              pyhd8ed1ab_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.10             pyhd8ed1ab_0    conda-forge
wiimatch                  0.3.2                    pypi_0    pypi
xarray                    2024.1.1           pyhd8ed1ab_0    conda-forge
xarray-einstats           0.7.0              pyhd8ed1ab_0    conda-forge
xerces-c                  3.2.5                hf393695_0    conda-forge
xorg-libxau               1.0.11               hb547adb_0    conda-forge
xorg-libxdmcp             1.1.3                h27ca646_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
zarr                      2.17.0             pyhd8ed1ab_0    conda-forge
zeromq                    4.3.5                h965bd2d_0    conda-forge
zfp                       1.0.1                ha8f4885_0    conda-forge
zict                      3.0.0              pyhd8ed1ab_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h53f4e23_5    conda-forge
zlib-ng                   2.0.7                h1a8c8d9_0    conda-forge
zstd                      1.5.5                h4f39d0f_0    conda-forge

Code of Conduct

[X] I agree to follow this project's Code of Conduct

taylorbell57 commented 5 months ago

Thanks for bringing this to our attention! Just to confirm, as far as you can tell from what outputs and plots were made, the fit looks like it converged well to a reasonable set of values and it is only this plot that is causing an issue? I think I've likely encountered this before as well - I'll see if I've already patched this in one of my in prep. pull requests.

rluquer commented 5 months ago

I'd say so, but it is hard to judge from the plots because the transit model + systematics or the data - systematics do not get drawn in the figures produced when isplots_S5=1

rluquer commented 5 months ago

Any news?

taylorbell57 commented 5 months ago

Sorry, I had a paper that needed to be resubmitted rapidly that is now taken care of. I'll try my best to resolve this before the end of the week.

taylorbell57 commented 5 months ago

I'd say so, but it is hard to judge from the plots because the transit model + systematics or the data - systematics do not get drawn in the figures produced when isplots_S5=1

Wait, do you mean that there are no model lines drawn on these figures? That's a different issue that what I thought was going on, but I think I know how to take care of that issue too. Can you confirm what reduced chi-squared value and scatter_mult value you're getting (should be very near 1.0 and >1.0, respectively); I ask since sometimes issues happen when scatter_mult goes negative which can happen with broad Normal priors.

taylorbell57 commented 5 months ago

But other issues have sometimes arisen for me where masked values don't play nicely with the GP code for reasons I don't fully understand

rluquer commented 5 months ago

In Figs510X:

5101: top data-only (no model); middle model-only (no data); bottom nothing
5102: top data-only (no model); middle nothing; bottom nothing
5103: data-only
5104: model-only

scatter_mult is 1.4+-0.01, no negative values (I've already avoided that with U-priors) reduced chi-squared ---

taylorbell57 commented 5 months ago

Thanks, that helps me narrow down the issue a lot!!

rluquer commented 5 months ago

So the fit apparently is done correctly, I've tried dynasty emcee and lsq. All break the code when isplots>=3, but when isplots=1 and the fit finishes, then I have those figures lacking the model and/or data

rluquer commented 5 months ago

the models are certainly missing the GP term for what I can see in Fig 5104

taylorbell57 commented 5 months ago

FYI, I was able to reproduce this bug with one of my own datasets today which is helping me narrow down the issue. Still working on the solution though

rluquer commented 5 months ago

Great news, thank you @taylorbell57. Keep me posted :)

kevin218 commented 5 months ago

Caleb Canas may have solved the problem. He says it works for him.

rluquer commented 5 months ago

Oh, this is great! Is there any solution that I can test in the meantime?

taylorbell57 commented 5 months ago

@rluquer, I just came up with a quick patch that works for me - you could try checking out the pull request above

kevin218 / Eureka