nannau / DoWnGAN

PyTorch/MLflow implementation of Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP) to perform single image super resolution (SISR) to downscale climate fields.
GNU General Public License v3.0
4 stars 1 forks source link

Implement CDO as preprocessing for the input fields. #13

Open nannau opened 1 year ago

nannau commented 1 year ago

Improve preprocessing pipeline. Look into using CDO because it is optimized for the standardization that we need to do before loading tensors into the training pipeline.

One option is nctoolkit https://nctoolkit.readthedocs.io/en/latest/

Also, look into saving the marginal distribution statistics used to standardize: i.e. the global mean and standard deviation into an additional NetCDF file so that metrics can be reported in the original units during training.