search

NVIDIA / modulus

Open-source deep-learning framework for building, training, and fine-tuning deep learning models using state-of-the-art Physics-ML methods
https://developer.nvidia.com/modulus
Apache License 2.0
836 stars 190 forks source link

CorrDiff: Better handling of the dataset attributes #415

Open mnabian opened 3 months ago

mnabian commented 3 months ago 
cfg.dataset.data_path = to_absolute_path(cfg.dataset.data_path)
    if hasattr(cfg, "global_means_path"):
        cfg.global_means_path = to_absolute_path(cfg.global_means_path)
    if hasattr(cfg, "global_stds_path"):
        cfg.global_stds_path = to_absolute_path(cfg.global_stds_path)

The first line should be moved to the CWB dataloader because this triggers an error when using a dataloader that doesn't have a data_path attribute. For example the other dataloader we are working on. And the global_means_path and global_stds_path should also be the responsibility of the dataloader.