481 Submitit Dev

[Pale-Blue-Dot-97]

Major Updates

• Closes #481
• Integrated the submitit plugin for hydra into minerva. This allows users on SLURM clusters to submit jobs and sweeps of jobs using the power of SLURM and hydra right from the command line.
• Added the MinervaSSL4EO dataset -- an adaption of the dataset defined in the SSL4EO-S12 repo. As part of this, the LMDB dataset functionality has also been integrated, along with the seasonal transforms integrated into the sample pairing.
• Major efficiencies in the code have been made by bypassing checks for infinities and NANs in modelio and calc_euc_dist which were discovered to be computationally expensive.

Other Improvements

• Users can now define log_rate in the global or task level params to define the frequency at which batches are logged to the logger. This may result in speed-ups from reducing the number of times loss.item() is called.
• Users can define what metrics to calculate in the SSLStepLogger

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 40.38462% with 186 lines in your changes missing coverage. Please review.

Project coverage is 91.98%. Comparing base (f35eaed) to head (38b39d4). Report is 436 commits behind head on 458-downstream-tasks.

:exclamation: Current head 38b39d4 differs from pull request most recent head 23191c1

Please upload reports for the commit 23191c1 to get more accurate results.

Files	Patch %	Lines
minerva/datasets/ssl4eos12.py	25.94%	137 Missing :warning:
minerva/modelio.py	22.22%	14 Missing :warning:
minerva/transforms.py	20.00%	12 Missing :warning:
minerva/datasets/paired.py	66.66%	10 Missing :warning:
minerva/utils/runner.py	62.50%	6 Missing :warning:
minerva/datasets/factory.py	16.66%	5 Missing :warning:
minerva/logger/steplog.py	85.71%	2 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## 458-downstream-tasks #484 +/- ## ======================================================== - Coverage 98.32% 91.98% -6.34% ======================================================== Files 38 41 +3 Lines 3931 4552 +621 ======================================================== + Hits 3865 4187 +322 - Misses 66 365 +299 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.