Primarily updates the metrics.py file to be more efficient and succinct. Included changes:
Removes some of the auto-naming of validation metric outputs, to make things cleaner.
_get_ranks wasn't being used in PosteriorCoverage and we were calculating ranks in the __call__ function
PosteriorCoverage now wraps PosteriorSamples and they use the same helper function to sample the whole test set (cleaner).
All helper functions of PosteriorCoverage now have the same input structure
PlotSinglePosterior and PosteriorCoverage now have a save_samples flag, which saves their (single) or (full) test samples to file
PlotSinglePosterior now has a seed attribute, to specify the random seed used to choose a test point.
All validation metrics now return a matplotlib figure or array of figures, in case people want to go in and tweak their properties later.
Adds matplotlib style loading. Now, we can specify a mpl style file with style_path in our ValidationRunner configuration, to create global styles for all validation plots.
Adds logging to anytime a validation metric is saved, so we know where it's going.
Adds fill_between error bars to _plot_coverage, instead of random sparse lines
Updates the docstrings of all functions so that they are accurate.
Fixed a bug wherein the DelfiWrapper was not saving the run names for later signatures.
Addresses the following tasks from Issue #88 :
Clean up validation output filenames
Add ability to specify matplotlib style or kwargs in validation metric plots
Primarily updates the
metrics.pyfile to be more efficient and succinct. Included changes:_get_rankswasn't being used inPosteriorCoverageand we were calculating ranks in the__call__functionPosteriorCoveragenow wrapsPosteriorSamplesand they use the same helper function to sample the whole test set (cleaner).PosteriorCoveragenow have the same input structurePlotSinglePosteriorandPosteriorCoveragenow have asave_samplesflag, which saves their (single) or (full) test samples to filePlotSinglePosteriornow has aseedattribute, to specify the random seed used to choose a test point.style_pathin our ValidationRunner configuration, to create global styles for all validation plots.fill_betweenerror bars to _plot_coverage, instead of random sparse linesAddresses the following tasks from Issue #88 :