Add Caching to Disk in Datapipe

[jacobbieker]

Pull Request

Description

This gives the option to cache to disk the outputs of some datapipes, and gives examples in the training pipeline on how to do so. This should help with compute intensive data pipelines.

Fixes #

How Has This Been Tested?

Unit tests and running it.

• [ ] Yes

Checklist:

• [ ] My code follows OCF's coding style guidelines
• [ ] I have performed a self-review of my own code
• [ ] I have made corresponding changes to the documentation
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] I have checked my code and corrected any misspellings

[codecov[bot]]

Codecov Report

Merging #182 (57ba8db) into main (86fa3de) will decrease coverage by 0.06%. The diff coverage is 8.33%.

:exclamation: Current head 57ba8db differs from pull request most recent head 4a21574. Consider uploading reports for the commit 4a21574 to get more accurate results

@@            Coverage Diff             @@
##             main     #182      +/-   ##
==========================================
- Coverage   77.79%   77.73%   -0.06%     
==========================================
  Files         124      124              
  Lines        5097     5106       +9     
==========================================
+ Hits         3965     3969       +4     
- Misses       1132     1137       +5     

Impacted Files	Coverage Δ
ocf_datapipes/training/metnet_pv_site.py	22.98% <8.33%> (-1.38%)	:arrow_down:

... and 1 file with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[jacobbieker]

This has already partly been done in #238