JuliaTrustworthyAI / LaplaceRedux.jl

Effortless Bayesian Deep Learning through Laplace Approximation for Flux.jl neural networks.
https://www.taija.org/LaplaceRedux.jl/
MIT License
39 stars 3 forks source link

Add a function to compute the average calibration in utils.jl #90

Closed pasq-cat closed 1 month ago

pasq-cat commented 4 months ago

i would like to try to add a function that plot the average level of calibration in utils.jl.

the function is defined in https://arxiv.org/pdf/1807.00263

pasq-cat commented 4 months ago

the end results should look like this for a miscalibrated network image and like this for a calibrated network image

pat-alt commented 4 months ago

Yes, nice idea! Have a look at https://github.com/devmotion/CalibrationErrors.jl for reference. It would be nice to have the basic evaluation function here, but the plotting in TaijaPlotting