icecube / pisa

Monte Carlo-based data analysis
http://icecube.github.io/pisa/
Apache License 2.0
19 stars 47 forks source link

Merging oscNext high stats analysis latest changes #686

Closed ts4051 closed 2 years ago

ts4051 commented 2 years ago

NOTE: This replaces PR #684. Code is the same, but commit history is simplified.

This is a merge of a number of changes implemented in recent months to support the oscNext high stats analyses.

Summary of changes:

1) Fix to bug in analysis fits where updates to params with multiple selections defined (e.g. mass orderings) are lost. This stopped sensitivity scans working for the numu disappearance analysis. The fix mostly involved removing taking copies of param objects, such as during the octant check.

2) Before scipy minimization begins, there is now a check to see if the initial params already yield a template that is identical to the data. If so, there is no need to minimize. This saves time, and avoids issues observed with the L-BFGS-B algorithm which can sometimes fail if the initial param values are exactly correct.

3) Added more metadata for failed fits.

4) Added smarter down-sampling of events, allowing breaking the events into arbitrary sized sub-populations, and selecting the desired sub-population. Used to create statistically independent templates with arbitrary statistics.

5) Added support for setting the tex member of the Param class via config files.

6) The grid.py stage (which creates fake input data on a grid, useful for making e.g. oscillograms) didn't work in PISA 4. This is now fixed.

7) Neutrino decoherence re-integrated into the nusquids.py stage )this has been broken by recent updates as part of the sterile analysis)

8) Some general performance improvements to the hypersurface code. No functional changes however, just better memory, file size, load times, error handling, etc. Also added additional data provenance.

9) Some ranges added to nufitv40.cfg.

10) A few general added/improved comments, etc.

11) Some more options exposed to user in KDE stage

philippeller commented 2 years ago

Great, Thanks for the PR @ts4051 and the review @atrettin ! Will merge this now