kratsg
commented
4 months ago Here's an example where one runs into this issue.
import pyhf
import json
import numpy as np
pyhf.set_backend("jax", pyhf.optimize.minuit_optimizer(verbose=0))
ws = pyhf.Workspace(json.load(open('simplified.json')))
model = ws.model()
data = ws.data(model)
n_samples = 3000
result = pyhf.infer.hypotest(
1.0,
data,
model,
init_pars=model.config.suggested_init(),
par_bounds=model.config.suggested_bounds(),
fixed_params=model.config.suggested_fixed(),
calctype="toybased",
return_expected_set=True,
ntoys=n_samples,
)This could maybe be improved with something like a better set of inits(?) but to first order, a more helpful or instructive error message would be nice.
Summary
When running fits, such as when doing toys, for example, one might run into a stack trace like so:
Certainly (and discussing with @alexander-held offline), the underlying issue which is a bit hard to parse is likely from this portion:
where the parameter of interest $\mu \in [0,100]$ but we have the uncertainty such that $\hat{\mu} = 4 \pm 94.68$. Now, in this case, the Hessian is not used, so whatever comes during Migrad to calculate the uncertainty is what we rely on. It's not necessarily clear exactly how it fails, but the likely culprit is that we are just too close to the bounds to estimate the uncertainty correctly.
There is a nice presentation from @alexander-held on improving the $\tilde{q}_\mu$ definition which could definitely be considered (https://indico.cern.ch/event/1336439/contributions/5629384/attachments/2734750/4755122/20231016_qmutilde.pdf).
However, regardless of this, we might as well provide some parsing since we know estimated distance is too large, to try and determine which of the parameters is having issues with error estimation at the end of the day, to give a better hint automatically.
Additional Information
No response
Code of Conduct