Earth-2 Model Intercomparison Project (MIP) is a python framework that enables climate researchers and scientists to inter-compare AI models for weather and climate.
Scoring with hdf5 files fails. This was caused by inconsistent handling of the batch dimension.
Previously data sources inserted this themselves so data_source[time].shape is (1, channel, lat, lon), but this is confusing and not how the hdf5.DataSource works. DataSources should just get (channel, lat, lon). I documented this in the base.DataSource and modified our data sources accordingly. I also removed some duplicate test files.
Checklist
[x] I am familiar with the Contributing Guidelines.
[x] New or existing tests cover these changes.
[x] The documentation is up to date with these changes.
[x] The CHANGELOG.md is up to date with these changes.
Earth-2 MIP Pull Request
Description
Scoring with hdf5 files fails. This was caused by inconsistent handling of the batch dimension.
Previously data sources inserted this themselves so
data_source[time].shape
is (1, channel, lat, lon), but this is confusing and not how the hdf5.DataSource works. DataSources should just get (channel, lat, lon). I documented this in the base.DataSource and modified our data sources accordingly. I also removed some duplicate test files.Checklist
Dependencies