Closed drewoldag closed 1 year ago
Now that PIT metrics have been moved from RAIL to qp, the code that evaluated the meta metrics has been refactored as well. A new method, calculate_pit_meta_metrics
does the work of calculating all the meta metrics. The user also has the option of calling each meta metric calculation individually.
The code can be seen here: https://github.com/LSSTDESC/qp/blob/main/src/qp/metrics/pit.py#L78
In the
PIT.evaluate
method this code will automatically evaluate the PITMetaMetrics that are decorated with the@PITMetaMetric
decorator. However, the decorator has a slight logic error.In the decorator in
pit.py
we're looking for aevaluate
method in the decorated class that has default arguments defined. However, only two of the metametrics have a default arugment. The others don't have input arguments that require default args - those metrics are not included.As it stands, only PITOutRate and PITAD are evaluated by default while KS and CvM are not. The
evaluate
methods of KS and CvM do not include arguments other than self.The action item here is to fix the logic in the decorator such that all the MetaMetrics are included as expected.