NVIDIA / earth2mip

Earth-2 Model Intercomparison Project (MIP) is a python framework that enables climate researchers and scientists to inter-compare AI models for weather and climate.
https://nvidia.github.io/earth2mip/
Apache License 2.0
187 stars 41 forks source link

Consistent shape for data sources #92

Closed nbren12 closed 10 months ago

nbren12 commented 10 months ago

Earth-2 MIP Pull Request

Description

Scoring with hdf5 files fails. This was caused by inconsistent handling of the batch dimension.

Previously data sources inserted this themselves so data_source[time].shape is (1, channel, lat, lon), but this is confusing and not how the hdf5.DataSource works. DataSources should just get (channel, lat, lon). I documented this in the base.DataSource and modified our data sources accordingly. I also removed some duplicate test files.

Checklist

Dependencies

nbren12 commented 10 months ago

/blossom-ci

nbren12 commented 10 months ago

/blossom-ci