Questions on simplified likelihood

[matthewfeickert]

This is a transfer of @sabinekraml's question on https://github.com/scikit-hep/pyhf/discussions/1689:

Hi, this does not concern pyhf proper but, as the subject says, simplified json files produced with the simplify tool. I'm interested in knowing

• why in the simplified likelihood the background expectations are replaced with the post-fit numbers; this eliminates any information about small deviations from SM expectations, relevant for e.g. BSM fits
• how correlated uncertainties are treated in simplify: if there are both correlated and uncorrelated nuisances, how are the final region-to-region correlations determined? The mathematical description would be helpful.

Thanks! Sabine

[matthewfeickert]

Following up on this question, @sabinekraml and @Ga0l made these slides and questions for the simplified likelihoods discussion of the publication of statistical models workshop.

cc @lukasheinrich

[eschanet]

@kratsg or @lukasheinrich might want to comment as well, but I'll have a shot at this already.

Not sure I follow the first point. Setting the pre-fit values of the simplified LH to be the post-fit ones of the full LH is exactly the point since we want to fix the background model to be what you get after running a background-only fit in the full LH (+ massively reducing the number of parameters).

By construction, there is no difference in how we handle correlations as compared to any other full LH. Remember, the numbers we plug into the simplified LH come from a full fit using the full LH -- i.e. considering the full list of NPs, channels and samples. The simplified LH itself (after the fit with the full LH) then only has a single nuisance parameter, i.e. here the total uncertainties on the background -- obtained by considering the full set of NPs and all correlations -- are considered to be fully correlated over all regions.

[sabinekraml]

I'm afraid I don't fully understand, but it's been a long day, including a long discussion session, and my brain needs a bit of air :-) In any case, perhaps you can have a look at https://indico.cern.ch/event/1088121/contributions/4592009/attachments/2343348/3995817/simplify-discussion.pdf

and we get together at some convenient time to talk via Zoom.

Cheers, Sabine

On Wed, Nov 10, 2021 at 7:05 PM Eric Schanet @.***> wrote:

@kratsg https://github.com/kratsg or @lukasheinrich https://github.com/lukasheinrich might want to comment as well, but I'll have a shot at this already.

Not sure I follow the first point. Setting the pre-fit values of the simplified LH to be the post-fit ones of the full LH is exactly the point since we want to fix the background model to be what you get after running a background-only fit in the full LH (+ massively reducing the number of parameters).

By construction, there is no difference in how we handle correlations as compared to any other full LH. Remember, the numbers we plug into the simplified LH come from a full fit using the full LH -- i.e. considering the full list of NPs, channels and samples. The simplified LH itself (after the fit with the full LH) then only has a single nuisance parameter, i.e. here the total uncertainties on the background -- obtained by considering the full set of NPs and all correlations -- are considered to be fully correlated over all regions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eschanet/simplify/issues/19#issuecomment-965603720, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG3ROOD4GKUXKU6OEUAN2O3ULKX77ANCNFSM5HYCAHYQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

--

---

Sabine Kraml - @.*** - www.kraml.net

LPSC 53 Av des Martyrs 38026 Grenoble France (+33)(0)4 76 28 40 52