torluca
commented
1 year ago Final report: we coded a first working version of work in RAIL. This consists of a new class for the ConditionPIT included in rail.evalutation.metrics, as well as a utils folder containing functions for training the local PIT. The new implemented code can be see in the branch "issue/228/calpreddistr" of RAIL. Many thanks to Bitrateep Dey for his help and availability.
Dey+22 statistical test in RAIL
Continuing the work started at the August DESC meeting on the implementation of calibrated predictive distributions (Dey+22) in rail.evaluate.
Contacts: Luca Tortorelli, Bitrateep Dey Day/Time: 20/10 from 10 am CET to 9 pm CET, 21/10 from 10 am CET to 7 pm CET Main communication channel: Slack GitHub repo: https://github.com/LSSTDESC/RAIL/tree/issue/228/calpreddistr Zoom room (if applicable): https://lmu-munich.zoom.us/j/99474383270?pwd=S0taTVZRR0pjNVA0QXk4YmcwaW5zUT09
Goals and deliverable
The goal of the sprint activity would be to keep working on the implementation of the code developed by Bitrateep Dey in rail.evaluate. This corresponds to the milestone: "Statistical tests like Dey et al. (2022) as part of the RAIL evaluate module" detailed in the GER-LMU in-kind workplan, together with the deliverable: "Documented contributions in the GitHub repository of RAIL."
Resources and skills needed
Familiarity with RAIL and qp, Dey+22 work.
Detailed description
The idea of the sprint is to continue the work of implementing the calibrated predictive distributions presented in Dey+22 into the rail.evaluate module. Bitrateep's code is publicly available and a first implementation of his work is already available in RAIL. Revision and optimisation of the implementation needs to be conducted.