mllam / neural-lam

Neural Weather Prediction for Limited Area Modeling
MIT License
64 stars 24 forks source link

Move dependencies to pyproject toml and create ci/cd install and import test #37

Open leifdenby opened 1 month ago

leifdenby commented 1 month ago

The aim of this PR is to move dependencies from requirements.txt to pyproject.toml (as has been proposed for v0.2.0 on the roadmap) and discussed on https://github.com/mllam/neural-lam/pull/27#issuecomment-2110001148.

Requirements:

leifdenby commented 1 month ago

I will copy here my motivation for allow using of pdm from https://github.com/mllam/neural-lam/pull/27#issuecomment-2117603643:

For someone not using pdm the process of setting up a new development environment would be:

  1. Clone repo git clone http://github.com/<username>/neural-lam and change directory cd neural-lam
  2. Optionally create your virtualenv with your favourite tool, maybe mkvirtualenv ... and then activate ...
  3. Install neural-lam editable with pip so that changes you make are reflected in the site-packages version (actually this is simply a link) pip install -e .
  4. Edit and run neural-lam: python -m neural_lam.train_model --config-file config.yaml
  5. Manually install a dependency with pip and edit pyproject.toml by hand, i.e. python -m pip install ... and edit the pyproject.toml file
  6. Stage, commit and push!

I like using a package manager like pdm (other options are pipenv, poetry, uv) because it makes the development process easier by 1) automatically updating package versions in pyproject.toml to ensure versions work together when I add a new package, 2) allowing me to "dry run" any dependency modification without clobbering my current env (this has hurt me quite a few times with pip) and 3) handling creating and activating virtuals envs

So with pdm the development process would be:

  1. Clone repo git clone http://github.com/<username>/neural-lam and change directory cd neural-lam
  2. Create virtual env pdm venv create and either activating it pdm venv activate ... or making it the default when using pdm run python ... with pdm use ...
  3. Install package pdm install
  4. Add dependency pdm add ... (this is where you can do pdm add --dry-run ... to see what would change before you install a package), or remove one with pdm remove .... You can also add dev dependencies separately (which won't be included in wheels) with pdm add --dev ... or dependency groups with pdm add --group visualisation matplotlib for example (if we didn't want visualisation tools installed by default)
  5. Edit and run neural-lam: python -m neural_lam.train_model --config-file config.yaml (if using activated virtualenv) or pdm run python -m neural_lam.train_model --config-file config.yaml
  6. Stage, commit and push!