mllam / neural-lam

Neural Weather Prediction for Limited Area Modeling
MIT License
64 stars 24 forks source link

Replace constants.py with data_config.yaml #31

Closed sadamov closed 1 month ago

sadamov commented 1 month ago

Summary This PR replaces the constants.py file with a data_config.yaml file. Dataset related settings can be defined by the user in the new yaml file. Training specific settings were added as additional flags to the train_model.py routine. All respective calls to the old files were replaced.

Rationale

Testing Both training and evaluation of the model were succesfully tested with the meps_example dataset.

Note @leifdenby Could you invite Thomas R. to this repo, in case he wanted to give his input on the yaml file? This PR should mostly serve as a basis for discussion. Maybe we should add more information to the yaml file as you outline in https://github.com/mllam/mllam-data-prep. I think we should always keep in mind how the repository will look like with realistic boundary conditions and zarr-archives as data-input.

This PR solves parts of https://github.com/joeloskarsson/neural-lam/issues/23

leifdenby commented 1 month ago

From a quick glance this looks simply amazing @sadamov! Thanks for doing this work. I will give a thorough review later today/tomorrow. Just tagging @SimonKamuk to have a read and give your thoughts too. I've added @ThomasRieutord to the organisation too. I'll also send Thomas an email so that he definitely sees the PR.

sadamov commented 1 month ago

I implemented most requested changes in the latest commit and requested one more review. From my side we are clear to merge. The latest changes were again tested for model training and evaluation.

leifdenby commented 1 month ago

Remember to update the changelog before you merge @sadamov! I think I would call this a new (very useful!) feature. A few things also change here (where constants are stored and thus how they are accessed in the code)

leifdenby commented 1 month ago

Hurraay! :partying_face:

joeloskarsson commented 1 month ago

Wonderful job with this @sadamov!