MC/calibration data statistical uncertainty nuisance parameters

[JelleAalbers]

Currently the statistical errors on a PDF for a given combination of parameters are assumed to be negligible. When you make a PDF from MC, you can often get to this happy point if you are patient.. but not when deriving a PDF from data.

It would be nice if there was an option to have a parameter to vary the expectation in each bin in each PDF used, and a corresponding Poisson term in the likelihood -- or at least on such parameter/erm for each bin of the total PDF. However:

• Minimizing these guys might be a tough cookie for the minimizer... the LHC folks have some special magic for this (Beeston-Barlow light) that may be worth looking into;
• This won't generalize to using KDEs as density estimators, for which we might have to do e.g. bootstrapping of the calibration/MC data;
• When you don't have enough counts in a bin you are in trouble anyway (what's the error on a bin in which you see no events? Surely not 0..)

[JelleAalbers]

@kdund and I implemented the Beeston-Barlow method for a single source with statistical uncertainties on the model PDF, generalized to the case where there are other sources present that do not have such statistical uncertainties. It turned out that the minimization with respect to the per-bin nuisance parameters can still be performed analytically in this case (even though the solution is an unsightly quadratic root).

The case of multiple sources will be more complicated; as no analytic solution is available, numerical methods would have to be used. For our present purposes the current implementation will be fine.