search

scikit-hep / pyhf

pure-Python HistFactory implementation with tensors and autodiff
https://pyhf.readthedocs.io/
Apache License 2.0
283 stars 83 forks source link

feat: Improve warnings for qmu and qmu_tilde for the set POI bounds #2390

Closed lorenzennio closed 10 months ago

lorenzennio commented 10 months ago

Description

When running 

model = pyhf.simplemodels.uncorrelated_background(signal=[5.0, 10.0], bkg=[50.0, 60.0], bkg_uncertainty=[5.0, 12.0])
pyhf.infer.hypotest(1.0, [53.0, 65.0] + model.config.auxdata, model, test_stat="qmu")

we get a warning

qmu test statistic used for fit configuration with POI bounded at zero.
Use the qmu_tilde test statistic (pyhf.infer.test_statistics.qmu_tilde) instead.

and vice versa for POIs not bounded at 0.

But this should also tell you to set test_stat="qtilde" or (test_stat="q").

Checklist Before Requesting Reviewer

Before Merging

For the PR Assignees:

* Add information to the user in the warning that provides them with the
  higher level pyhf.infer APIs kwarg to set the correct test statistic.
codecov[bot] commented 10 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (4d215a7) 96.29% compared to head (efe4ba2) 98.28%. Report is 3 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #2390 +/- ## ========================================== + Coverage 96.29% 98.28% +1.98% ========================================== Files 69 69 Lines 4539 4539 Branches 648 803 +155 ========================================== + Hits 4371 4461 +90 + Misses 128 45 -83 + Partials 40 33 -7 ``` | [Flag](https://app.codecov.io/gh/scikit-hep/pyhf/pull/2390/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scikit-hep) | Coverage Δ | | |---|---|---| | [contrib](https://app.codecov.io/gh/scikit-hep/pyhf/pull/2390/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scikit-hep) | `97.86% <ø> (?)` | | | [doctest](https://app.codecov.io/gh/scikit-hep/pyhf/pull/2390/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scikit-hep) | `60.71% <ø> (?)` | | | [unittests-3.10](https://app.codecov.io/gh/scikit-hep/pyhf/pull/2390/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scikit-hep) | `96.29% <ø> (ø)` | | | [unittests-3.11](https://app.codecov.io/gh/scikit-hep/pyhf/pull/2390/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scikit-hep) | `96.29% <ø> (?)` | | | [unittests-3.8](https://app.codecov.io/gh/scikit-hep/pyhf/pull/2390/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scikit-hep) | `96.32% <ø> (?)` | | | [unittests-3.9](https://app.codecov.io/gh/scikit-hep/pyhf/pull/2390/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scikit-hep) | `96.34% <ø> (?)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scikit-hep#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

matthewfeickert commented 9 months ago

@lorenzennio This fix is now out in pyhf v0.7.6.