Implement simple model checkpointing

[nmichlo]

Hi @meffmadd, thank you so much for this contribution. It looks great! 😁🎉

Before we merge, please may you update the checkpointing in the configs as some tests are currently failing and for documentation purposes?

• Both in the main config (possibly defaulting to false to maintain current behavior)
  • https://github.com/nmichlo/disent/blob/8dee583993d582729928984640930a4e92e65de6/experiment/config/config.yaml#L44-L47
• And copying this to the test config in the corresponding location (defaulting to true so we know the checkpoint behavior works).
  • https://github.com/nmichlo/disent/blob/8dee583993d582729928984640930a4e92e65de6/experiment/config/config_test.yaml

On that note, we might want to just add the checkpointing to the pytorch lightning train step, using the same hook. Then possibly at the end, load the checkpoint after everything to make sure it works.

• https://github.com/nmichlo/disent/blob/8dee583993d582729928984640930a4e92e65de6/tests/test_frameworks.py#L98-L126

[meffmadd]

Hi, I couldn't work on it this weekend but will start now and fix the configs so that the tests work again. I will also add a test case that tests the behavior.

[codecov[bot]]

Codecov Report

Base: 70.01% // Head: 70.04% // Increases project coverage by +0.02% :tada:

Coverage data is based on head (28d0e95) compared to base (8dee583). Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #37 +/- ## ========================================== + Coverage 70.01% 70.04% +0.02% ========================================== Files 135 135 Lines 7531 7538 +7 ========================================== + Hits 5273 5280 +7 Misses 2258 2258 ``` | [Impacted Files](https://codecov.io/gh/nmichlo/disent/pull/37?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Nathan) | Coverage Δ | | |---|---|---| | [disent/frameworks/\_ae\_mixin.py](https://codecov.io/gh/nmichlo/disent/pull/37/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Nathan#diff-ZGlzZW50L2ZyYW1ld29ya3MvX2FlX21peGluLnB5) | `90.00% <100.00%> (+1.32%)` | :arrow_up: | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Nathan). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Nathan)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[nmichlo]

Great work! Thank you so much for making these changes!