search

gwastro / pycbc

Core package to analyze gravitational-wave data, find signals, and study their parameters. This package was used in the first direct detection of gravitational waves (GW150914), and is used in the ongoing analysis of LIGO/Virgo data.
http://pycbc.org
GNU General Public License v3.0
315 stars 349 forks source link

hierarchical statmap modifications #1154

Closed ahnitz closed 7 years ago

ahnitz commented 8 years ago
stevereyes01 commented 8 years ago

I've been traveling since after the F2F, but I'm back in Syracuse and working on it. I'll post updates/questions as the code progresses.

duncan-brown commented 7 years ago

@stevereyes01 @ahnitz I assume we're not worried about this for the first ER10 release?

stevereyes01 commented 7 years ago

@spxiwh @ahnitz So I think I've got a working version of hierarchical coinc_statmap. Although, I still need to clean up parts of the code so that it's readable, better documented, and so that it's more Pythonic. I would like to make some plots to double check that everything looks sane. I can do that by hand, but coming up with a way for doing that within the workflow might require some further planning

The next question is how do we output the information from this code? Since there's a different inclusive-background generated by each loudest-event gravitational wave that gets removed in hierarchical removal, writing the FARs and FAPs to an output file as currently implemented seems problematic (or at least something I need to devote more time to).

The way that I have the code right now, it hierachically removes foreground triggers and the associated background triggers from time slides until there is no longer a foreground trigger that is louder than the inclusive background. Duncan suggested that I run this code to some cut-off point set by the user (say after 5 hierarchical removals) so that we don't accidentally end up with tons of hdf files or plots.

Does it sound like a good idea in the workflow to plot the regular plots that depend on the statmap codes, and have a separate tab called "Hierarchical Removal" that contains the plots/tables for every set of inclusive backgrounds made after a removal? For example, the first set of plots will be with GW150914, then the second set of plots will be with GW150914 removed (showing the significance of GW151226), and finally since GW151226 is louder than the background it will just show LVT151012 and the inclusive and exclusive backgrounds should be the same.

Let me know what you all think.

ahnitz commented 7 years ago

Steve, the FAR column in the output should just be the heirarchical far. The exclusive_far stays as it is currently, no need to make anything more complicated. Let's see you run this over the full-01 run and we'll see if any additional plots are needed. I don't really think so though, except for some review testing, but not normally.

On Thu, Dec 1, 2016 at 10:15 PM, Steven Reyes notifications@github.com wrote:

@spxiwh https://github.com/spxiwh @ahnitz https://github.com/ahnitz So I think I've got a working version of hierarchical coinc_statmap. Although, I still need to clean up parts of the code so that it's readable, better documented, and so that it's more Pythonic. I would like to make some plots to double check that everything looks sane. I can do that by hand, but coming up with a way for doing that within the workflow might require some further planning

The next question is how do we output the information from this code? Since there's a different inclusive-background generated by each loudest-event gravitational wave that gets removed in hierarchical removal, writing the FARs and FAPs to an output file as currently implemented seems problematic (or at least something I need to devote more time to).

The way that I have the code right now, it hierachically removes foreground triggers and the associated background triggers from time slides until there is no longer a foreground trigger that is louder than the inclusive background. Duncan suggested that I run this code to some cut-off point set by the user (say after 5 hierarchical removals) so that we don't accidentally end up with tons of hdf files or plots.

Does it sound like a good idea in the workflow to plot the regular plots that depend on the statmap codes, and have a separate tab called "Hierarchical Removal" that contains the plots/tables for every set of inclusive backgrounds made after a removal? For example, the first set of plots will be with GW150914, then the second set of plots will be with GW150914 removed (showing the significance of GW151226), and finally since GW151226 is louder than the background it will just show LVT151012 and the inclusive and exclusive backgrounds should be the same.

Let me know what you all think.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ligo-cbc/pycbc/issues/1154#issuecomment-264297325, or mute the thread https://github.com/notifications/unsubscribe-auth/ACGrRkBdOmEKtw04MVdVFtW5KfQQDlFUks5rDzjigaJpZM4KZ_7S .

-- Dr. Alexander Nitz Max Planck Institute for Gravitational Physics (Albert Einstein Institute) Callinstrasse 38 D-30167 Hannover, Germany Tel: +49 511 762-17097

stevereyes01 commented 7 years ago

Alright, thanks Alex, I'll do some reviewing of everything and hopefully have some plots over the weekend.

stevereyes01 commented 7 years ago

This got merged, so I'm closing this. See https://github.com/ligo-cbc/pycbc/pull/1328