MNGuenther / allesfitter

allesfitter is a convenient wrapper around the packages ellc (light curve and RV models), dynesty (static and dynamic nested sampling) emcee (Markov Chain Monte Carlo sampling) and celerite (Gaussian Process models).
MIT License
62 stars 36 forks source link

Phase plot containing unlikely models #79

Open YasminDavis opened 3 weeks ago

YasminDavis commented 3 weeks ago

Hi,

When I plot results from 'allesfitter' I keep getting poorly-fitting models included in the plot. Both RV and photometry data is being fit together. From my understanding, it is incredibly unlikely to see these models in the plots as they should not be included in the posterior samples. Am I missing a step when plotting in order to make sure I am vetting these models?

I am running 'allesfitter' version 1.2.8 on a target. I am plotting the output from 'allesfitter' from sampling), and even with a low number of 'Nsamples' (e.g. 20 or 200) I find multiple models that should have low probability.

I have tried my own plotting routine as well as the one in the example below, but I get the same result

#------------------------------------------------------------------------------
# Photometry
#------------------------------------------------------------------------------
#::: iterate over all plot styles
for style in ['full']:

    #::: set up the figure
    fig, axes = plt.subplots(2, 1, figsize=(6,6), gridspec_kw={'height_ratios': [3,1]}, sharex=True, Nsamples=200)
    fig.subplots_adjust(hspace=0)

    #::: alles.plot(...) data and model
    alles.plot('Leonardo','b',style,ax=axes[0])
    axes[0].set_title('Leonardo, '+style)

    #::: alles.plot(...) residuals
    alles.plot('Leonardo','b',style+'_residuals',ax=axes[1])
    axes[1].set_title('')

    fig.savefig('Leonardo_'+style+'.pdf', bbox_inches='tight')

github_issue_plot.pdf

MNGuenther commented 4 days ago

HI @YasminDavis - it's hard to look into this without seeing all your files (and I unfortunately cannot promise that I would have time to dig into them).

My gut feeling is that this seems a lot like plotting samples from a non-converged fit.

Is your fit using MCMC or Nested Sampling? I would guess MCMC, in which case I would guess the settings' number of walkers, length of burn steps, and total steps are too small to reach convergence.

Please let me know if this helps to get back on track, and if we can close this issue.