ENSTA-U2IS-AI / torch-uncertainty

Open-source framework for uncertainty and deep learning models in PyTorch :seedling:
https://torch-uncertainty.github.io
Apache License 2.0
287 stars 19 forks source link

:sparkles: Mixup variants + Cross validation + Temperature scaling in routines #57

Closed qbouniot closed 11 months ago

qbouniot commented 11 months ago
codecov[bot] commented 11 months ago

Codecov Report

Attention: 201 lines in your changes are missing coverage. Please review.

Comparison is base (99c935e) 97.95% compared to head (890ccce) 92.94%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## dev #57 +/- ## ========================================== - Coverage 97.95% 92.94% -5.01% ========================================== Files 71 74 +3 Lines 3610 3868 +258 Branches 428 473 +45 ========================================== + Hits 3536 3595 +59 - Misses 32 220 +188 - Partials 42 53 +11 ``` | [Flag](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS) | Coverage Δ | | |---|---|---| | [cpu](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS) | `92.94% <45.52%> (-5.01%)` | :arrow_down: | | [pytest](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS) | `92.94% <45.52%> (-5.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS) | Coverage Δ | | |---|---|---| | [...rch\_uncertainty/baselines/classification/resnet.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvYmFzZWxpbmVzL2NsYXNzaWZpY2F0aW9uL3Jlc25ldC5weQ==) | `98.14% <100.00%> (ø)` | | | [torch\_uncertainty/baselines/classification/vgg.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvYmFzZWxpbmVzL2NsYXNzaWZpY2F0aW9uL3ZnZy5weQ==) | `97.56% <100.00%> (ø)` | | | [...uncertainty/baselines/classification/wideresnet.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvYmFzZWxpbmVzL2NsYXNzaWZpY2F0aW9uL3dpZGVyZXNuZXQucHk=) | `98.07% <100.00%> (ø)` | | | [torch\_uncertainty/baselines/regression/mlp.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvYmFzZWxpbmVzL3JlZ3Jlc3Npb24vbWxwLnB5) | `97.05% <100.00%> (ø)` | | | [torch\_uncertainty/datamodules/uci\_regression.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvZGF0YW1vZHVsZXMvdWNpX3JlZ3Jlc3Npb24ucHk=) | `100.00% <100.00%> (ø)` | | | [...ainty/post\_processing/calibration/matrix\_scaler.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvcG9zdF9wcm9jZXNzaW5nL2NhbGlicmF0aW9uL21hdHJpeF9zY2FsZXIucHk=) | `100.00% <ø> (ø)` | | | [...\_uncertainty/post\_processing/calibration/scaler.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvcG9zdF9wcm9jZXNzaW5nL2NhbGlicmF0aW9uL3NjYWxlci5weQ==) | `100.00% <ø> (ø)` | | | [.../post\_processing/calibration/temperature\_scaler.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvcG9zdF9wcm9jZXNzaW5nL2NhbGlicmF0aW9uL3RlbXBlcmF0dXJlX3NjYWxlci5weQ==) | `100.00% <ø> (ø)` | | | [...ainty/post\_processing/calibration/vector\_scaler.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvcG9zdF9wcm9jZXNzaW5nL2NhbGlicmF0aW9uL3ZlY3Rvcl9zY2FsZXIucHk=) | `100.00% <ø> (ø)` | | | [torch\_uncertainty/routines/regression.py](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS#diff-dG9yY2hfdW5jZXJ0YWludHkvcm91dGluZXMvcmVncmVzc2lvbi5weQ==) | `89.26% <100.00%> (ø)` | | | ... and [12 more](https://app.codecov.io/gh/ENSTA-U2IS/torch-uncertainty/pull/57?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ENSTA-U2IS) | |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

o-laurent commented 11 months ago

Ouch, the coverage :rofl:

o-laurent commented 11 months ago

@qbouniot, could you start another PR using a branch that is not called main (but synced with main)? Linked to #58.