FPP figures/results

[timothydmorton]

OK, here are figures relevant to the FPP section. Working on text now.

Predicted numbers of signals to be found in this survey:

And typical shapes of those synthetic signals, compared to the observed candidate shapes:

It might not be necessary to include the first figure, as the information can be easily summarized in a table or in the text.

Using the mean number of synthetic detections as the prior and the shape distributions for the likelihood for each hypothesis I calculate the probability for each candidate to be EB, BEB, and planet. In the sample of 16, the expected number of EBs is 2.4, BEBs is 0.2, and planets 13.4. Two of the EBs are the two big ones 9306307 and 10602068 (mostly because my simulations only generate planets within the 0.1 to 1 Rjup bin). I compute 4754460 to have about a 3% chance of being a EB or BEB.

[dfm]

Awesome. Thanks so much for this!

I can't run your notebook without the "bgstars" file so can I ask for a few changes?

• add an interpolation="nearest" argument to the imshow call in plot_kde
• change the plot style for the red Xs. I'd probably prefer .g
• use \log_{10} instead of log for the y-axis – I'm not really an astronomer so log still means ln to me :-)
• would it be possible to add the error bars from fits.csv? There should be *_uncert_* columns in the file with obvious meanings. They might be too small to see, I guess but, if not, it would be cool to include them
• do you want to include a table of FPPs in the paper? If so, can you save the pandas DataFrame to a csv file in /results and I'll add it to the table.

Again. This is awesome. Thanks!

[timothydmorton]

Sure can do all this. I'll also do some more organization so that all my calculations are easily reproducible. On Fri, Jul 15, 2016 at 01:38 Dan Foreman-Mackey notifications@github.com wrote:

Awesome. Thanks so much for this!

I can't run your notebook without the "bgstars" file so can I ask for a few changes?

• add an interpolation="nearest" argument to the imshow call in plot_kde
• change the plot style for the red Xs. I'd probably prefer .g
• use \log_{10} instead of log for the y-axis – I'm not really an astronomer so log still means ln to me :-)
• would it be possible to add the error bars from fits.csv? There should be uncert columns in the file with obvious meanings. They might be too small to see, I guess but, if not, it would be cool to include them
• do you want to include a table of FPPs in the paper? If so, can you save the pandas DataFrame to a csv file in /results and I'll add it to the table.

Again. This is awesome. Thanks!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dfm/peerless/issues/11#issuecomment-232895014, or mute the thread https://github.com/notifications/unsubscribe-auth/ABzr2wf6Q_F_rRkebzZnZUOmNGcSMi73ks5qV0cfgaJpZM4JMvOL .

[timothydmorton]

OK, new plot. The error bars are too small to see, with the exception of the longest-duration signal, where the duration uncertainty is barely visible, so I nixed it.

I've also re-organized the predictions notebook, and it should now be fully reproducible.

[ericagol]

What does the grey scale indicate?

Eric Agol Astronomy Professor University of Washington

On Jul 15, 2016, at 4:50 PM, Timothy Morton notifications@github.com wrote:

OK, new plot. The error bars are too small to see, with the exception of the longest-duration signal, where the duration uncertainty is barely visible, so I nixed it.

I've also re-organized the predictions notebook, and it should now be fully reproducible.

― You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[timothydmorton]

Probability density of these two shape parameters for each scenario, based on the simulations.

[ericagol]

Do they have the same normalization?

Eric Agol Astronomy Professor University of Washington

On Jul 15, 2016, at 5:18 PM, Timothy Morton notifications@github.com wrote:

Probability density of these two shape parameters for each scenario, based on the simulations.

― You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[timothydmorton]

Each individual greyscale may not have the exact same normalization; however, each panel represents an individually normalized 2d pdf... is this confusing to interpret? I'm so used to making these plots I don't even think about it.

[ericagol]

The BEB & Planet overlap in this space, so I was trying to assess the relative probability of the two.

Eric Agol Astronomy Professor University of Washington

On Jul 15, 2016, at 5:33 PM, Timothy Morton notifications@github.com wrote:

Each individual greyscale may not have the exact same normalization; however, each panel represents an individually normalized 2d pdf... is this confusing to interpret? I'm so used to making these plots I don't even think about it.

― You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[timothydmorton]

The relative probability comes from the Nobserved histograms I posted above, where I predict the absolute numbers of detections. On Fri, Jul 15, 2016 at 15:44 ericagol notifications@github.com wrote:

The BEB & Planet overlap in this space, so I was trying to assess the relative probability of the two.

Eric Agol Astronomy Professor University of Washington

On Jul 15, 2016, at 5:33 PM, Timothy Morton notifications@github.com wrote:

Each individual greyscale may not have the exact same normalization; however, each panel represents an individually normalized 2d pdf... is this confusing to interpret? I'm so used to making these plots I don't even think about it.

― You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dfm/peerless/issues/11#issuecomment-233086513, or mute the thread https://github.com/notifications/unsubscribe-auth/ABzr2x8Is-OKkoNKVxjiWj4O3ktDygbbks5qWA09gaJpZM4JMvOL .

[dfm]

It might be interesting to see how the plot looks if we put them on the same vscale. There might not be enough contrast but if there is, it might be better. I'm happy to defer to your choice on that though if we just explain it in words.

[dfm]

I've added references to your FP section and added the FPPs to the parameter table. The only reference that I'm missing is the one exosyspop.

[timothydmorton]

I was planning to submit to ASCL for the reference, so we won't have the ID until it's on arxiv I think? I'd like to do a bit of documentation before that though, so maybe it's best to just have a footnote reference to the repo for now. On Sat, Jul 16, 2016 at 03:50 Dan Foreman-Mackey notifications@github.com wrote:

I've added references to your FP section and added the FPPs to the parameter table. The only reference that I'm missing is the one exosyspop.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/dfm/peerless/issues/11#issuecomment-233124865, or mute the thread https://github.com/notifications/unsubscribe-auth/ABzr23-k29du2K02sxck0y712j0SLYVsks5qWLd1gaJpZM4JMvOL .

[dfm]

Cool. Done.

[ericagol]

@timothydmorton @dfm

I had some questions I meant to ask about the false-alarm plots that puzzled me when I saw them (but I forgot to ask at the time):

1). What do you make of the points in your model at small depth? It looks like there are ~2-5 points at small depth which lie below at depth of 10^{-2.75}, which is below (or on the outskirts of) the significant probability region of any of the three simulated populations. Is the cutoff in your distribution towards small depth in your model due to a drop off in S/N? Or could it be due to a decline in small planet frequency? Could Dan's survey more sensitive to small planets than expected?

2). The two points at long duration are also puzzling as there isn't much probability in your models there as well: what do you make of those? Could they be large-radius stars that have snuck through the target selection process? For example, a radius that is large by a factor of ~3 would move your population of EBs from ~10^{-2.25} and duration of ~0.5 days to about where those to two points lie, so I'm wondering if these could be sub-giant EBs that snuck through Huber's analysis as MS stars. There is a bias that these would have a higher transit/eclipse probability, and thus would be over-represented in the observed sample.

3). Your FAP calculations take into account the relative probability of the the three populations, but I'm curious whether there is a way to compare to the overall distribution of observed depth/duration. Is there some way to better match the observed population by, say, adding some sub-giants or adding in a population of smaller planets?

[dfm]

I don't think we need to go into this for this paper. The simulations are approximate – they don't even use a real detection efficiency function! To look at a match between the distributions like you're asking for in point 3, we'd have to do the proper ABC inference that I would like to do some day but it's definitely way beyond the scope of what we're trying to do here.

[ericagol]

But you've simulated the real detection function with your planet injections, so you should at least have the data already in hand to make the right hand panel plot for your planet sample (with the inferred parameters), and compare it to the observed sample to see if the depth/duration distribution is similar (i.e. covers the same region of parameter space)... for the next paper :)

[timothydmorton]

These are very good questions, Eric, and I agree that ideally we'd like to see a better match between the simulated transiting planet shape distribution and what we see... and it's a good idea to make sure to account for subgiants when we do this.