Add Threadpool Map to more datapipes for speed ups

[jacobbieker]

Pull Request

Description

This adds threadpool mapping to the Irradiance loading. Also adds ability to add threadpools for other parts of the code.

Fixes #180

How Has This Been Tested?

Unit tests

• [x] Yes

Checklist:

• [ ] My code follows OCF's coding style guidelines
• [ ] I have performed a self-review of my own code
• [ ] I have made corresponding changes to the documentation
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] I have checked my code and corrected any misspellings

[codecov[bot]]

Codecov Report

Merging #181 (e567b7d) into main (b744c18) will increase coverage by 0.01%. The diff coverage is 83.05%.

:exclamation: Current head e567b7d differs from pull request most recent head cf6a53d. Consider uploading reports for the commit cf6a53d to get more accurate results

@@            Coverage Diff             @@
##             main     #181      +/-   ##
==========================================
+ Coverage   77.85%   77.87%   +0.01%     
==========================================
  Files         123      124       +1     
  Lines        5054     5094      +40     
==========================================
+ Hits         3935     3967      +32     
- Misses       1119     1127       +8     

Impacted Files	Coverage Δ
ocf_datapipes/training/pseudo_irradience.py	16.52% <25.00%> (+0.78%)	:arrow_up:
ocf_datapipes/utils/parallel.py	85.71% <85.71%> (ø)
..._datapipes/transform/xarray/metnet_preprocessor.py	92.75% <96.66%> (+0.69%)	:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more