To make batches more interperable and easier to debug/plot/etc we want to be able to save them out to NetCDF files rather than pytorch tensors.
This would help with making visualizations easier.
Possible Implementation
Add a datapipe/function that combines the multiple xarray data arrays and datasets into a single xarray dataset, with the coordinates and dimensions of the original datasets being prepended with what modality they are, similar to in the configuration.
Example: satellite coordinates are stored as "x_geostationary" would become "sat_x_geostationary"
A second function would undo those combinations back into the set of dataarrays and datasets.
To make batches more interperable and easier to debug/plot/etc we want to be able to save them out to NetCDF files rather than pytorch tensors.
This would help with making visualizations easier.
Possible Implementation
Add a datapipe/function that combines the multiple xarray data arrays and datasets into a single xarray dataset, with the coordinates and dimensions of the original datasets being prepended with what modality they are, similar to in the configuration.
Example: satellite coordinates are stored as "x_geostationary" would become "sat_x_geostationary"
A second function would undo those combinations back into the set of dataarrays and datasets.