Metric analysis - Githubissues

aimalz / qp

Quantile Parametrization for probability distribution functions module

MIT License

3 stars 9 forks source link

Metric analysis #57

Closed aimalz closed 7 years ago

aimalz commented 7 years ago

read in parametrized catalogs via #56
loop over N_metrics, N_floats, N_parametrizations
plot the results

drphilmarshall commented 7 years ago

Let's make a new notebook, data_analysis and use the Ensemble class to analyze all of Melissa's PDFs.

aimalz commented 7 years ago

I'm going to use this issue to group together issues for specific metrics, namely #47, #49, and #42. As qp will be the backend for the LSST-DESC PZ DC1 paper, it will need to have the following metrics implemented in the qp.Ensemble class:

[x] RMSE (#9)
[x] KLD (#9)
[ ] KS/AD/CvM (#47)
[ ] QQ (#42) (Is the plan still to do all of these @sschmidt23?) Some of these may require qp to have some notion of the true redshift, which is also a requirement of #53 (which in turn suggests #49).

aimalz commented 7 years ago

Update: I'm pretty happy with the status of issue/57/analysis and will run something like the analysis notebook on NERSC as soon as it's back on line.