dfm / tess-atlas

MIT License
9 stars 8 forks source link

Some initial "optimized" guesses looks very different from posteriors #201

Closed avivajpeyi closed 2 years ago

avivajpeyi commented 2 years ago

Hmm i wish I had added the TOI number in this...

I am a bit concerned that the initial fit is so different from the posterior for these TOIs.

Screen Shot 2022-06-03 at 2 24 26 pm Screen Shot 2022-06-03 at 2 24 35 pm Screen Shot 2022-06-03 at 2 24 43 pm Screen Shot 2022-06-03 at 2 25 00 pm Screen Shot 2022-06-03 at 2 25 09 pm Screen Shot 2022-06-03 at 2 25 23 pm Screen Shot 2022-06-03 at 2 25 40 pm

Check out this pdf for ~1.5K phase plots

avivajpeyi commented 2 years ago

Its bizarre that the initial fit is so off. Some things to check:

avivajpeyi commented 2 years ago

Focusing on TOI 2164

NOTE: initial fit == initial optimized parameters' fit

The initial fit is not in the 1,2,3-sigma of the posteriors.

Screen Shot 2022-06-08 at 8 57 40 am Screen Shot 2022-06-08 at 8 58 53 am

initial fit plotted on the data

The signal has an S/R~ 12 --> maybe this is the issue?

Screen Shot 2022-06-08 at 8 59 11 am Screen Shot 2022-06-08 at 9 03 12 am

Plot of the initial fit on the prior:

Initial fit in orange

Screen Shot 2022-06-08 at 9 04 59 am Screen Shot 2022-06-08 at 9 10 35 am Screen Shot 2022-06-08 at 9 12 18 am

What is going on with the noise params??

avivajpeyi commented 2 years ago

Plotting histograms of posterior + prior samples with initial fit

Ok, something really strange with noise parameters.... how are we getting a posterior outside our prior bounds?

Screen Shot 2022-06-08 at 9 21 05 am

Ok, I guess the tmax and Rp/Rstar results could make sense -- the posterior is inside the prior, the initial guess is kind of close to the posterior? But maybe our initial fit can be better?

Screen Shot 2022-06-08 at 9 25 32 am Screen Shot 2022-06-08 at 9 30 50 am
avivajpeyi commented 2 years ago

Optimization code + logs

theta = model.test_point
theta1 = pmx.optimize(theta, [jitter], **kwargs) # 1 noise param
theta2 = pmx.optimize(theta1, [b, r, dur], **kwargs) # impact param, Rp/Rs, transit duration
theta3 = pmx.optimize(theta2, [rho, sigma, jitter], **kwargs) # GP rho, GP sigma, jitter
theta4 = pmx.optimize(theta3, [u, f0], **kwargs) # limb darkening + baseline flux
theta5 = pmx.optimize(theta4,  [tmax_1], **kwargs) # transit timing
2022-06-08 23:42:34,204 - TESS-ATLAS - INFO - Optimizing sampling starting point
optimizing logp for variables: [jitter]
logp: -40938.29 -> -39062.89

optimizing logp for variables: [b, r, dur]
logp: -39062.89 -> -38961.56

optimizing logp for variables: [rho, sigma, jitter]
logp: -38961.56 -> -38839.13

optimizing logp for variables: [u, f0]
logp: -38839.13 -> -38838.41

optimizing logp for variables: [tmax_1]
logp: -38838.41 -> -38818.97

2022-06-08 23:45:24,923 - TESS-ATLAS - INFO - Optimization complete! (logp: -40928.32 -> -38742.42)
Screen Shot 2022-06-08 at 10 22 42 am

Looking at the theta5 'optimized' fit, it looks like

  1. t0 is correct,
  2. f0 is correct,
  3. transit duration is incorrect
  4. transit depth is incorrect

(code to generate plot init_lc_plotter.py.zip)

avivajpeyi commented 2 years ago
avivajpeyi commented 2 years ago
theta = model.test_point
theta1 = pmx.optimize(theta, [jitter], **kwargs) # 1 noise param
theta2 = pmx.optimize(theta1, [b, r, dur], **kwargs) # impact param, Rp/Rs, transit duration
theta3 = pmx.optimize(theta2, [rho, sigma, jitter], **kwargs) # GP rho, GP sigma, jitter
theta4 = pmx.optimize(theta3, [u, f0], **kwargs) # limb darkening + baseline flux
theta5 = pmx.optimize(theta4,  [tmax_1], **kwargs) # transit timing
theta6 = pmx.optimize(theta5, planet_params[0], **kwargs) # Rp/Rs
theta7 = pmx.optimize(theta6, [planet_params[1], period_params[0]], **kwargs) # duration, tmax
theta8 = pmx.optimize(theta7, [*planet_params, *noise_params, *period_params], **kwargs) # all params

A bit better! logp: -40928.32 -> -38714.38

download

@dfm -- do you have any thoughts on this?

avivajpeyi commented 2 years ago

Fixed by looping over optimization step