search

JuliaTrustworthyAI / ConformalPrediction.jl

Predictive Uncertainty Quantification through Conformal Prediction for Machine Learning models trained in MLJ.
https://juliatrustworthyai.github.io/ConformalPrediction.jl/
MIT License
135 stars 12 forks source link

62 full module for conformal training #81

Closed pat-alt closed 1 year ago

pat-alt commented 1 year ago

Original issue #55

Full training hinges on support for differentiable sorting (see discourse). I may merge a preliminary version anyway that already makes the smooth penalties available for downstream tasks.

pat-alt commented 1 year ago

Looks like using InferOpts.jl is the way to go: https://axelparmentier.github.io/InferOpt.jl/stable/tutorial/

codecov-commenter commented 1 year ago

Codecov Report

Merging #81 (88817cc) into main (5013ca1) will decrease coverage by 7.94%. The diff coverage is 1.13%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##             main      #81      +/-   ##
==========================================
- Coverage   83.81%   75.87%   -7.94%     
==========================================
  Files          15       20       +5     
  Lines         803      887      +84     
==========================================
  Hits          673      673              
- Misses        130      214      +84
Files Changed Coverage Δ
src/ConformalPrediction.jl 100.00% <ø> (ø)
...c/conformal_models/ConformalTraining/classifier.jl 0.00% <0.00%> (ø)
...dels/ConformalTraining/inductive_classification.jl 0.00% <0.00%> (ø)
...l_models/ConformalTraining/inductive_regression.jl 0.00% <0.00%> (ø)
src/conformal_models/ConformalTraining/losses.jl 0.00% <0.00%> (ø)
...rc/conformal_models/ConformalTraining/regressor.jl 0.00% <0.00%> (ø)
src/conformal_models/ConformalTraining/training.jl 0.00% <0.00%> (ø)
src/conformal_models/conformal_models.jl 92.30% <ø> (ø)
src/conformal_models/plotting.jl 88.33% <ø> (-0.10%) :arrow_down:
src/evaluation/measures.jl 78.26% <0.00%> (-16.48%) :arrow_down:
... and 1 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

pat-alt commented 1 year ago

This is still unfinished, but I'll merge because one of my research projects currently depends on this and I want to link it to a version that's on the registry.