NOAA-GSL / ExascaleWorkflowSandbox

Other
2 stars 2 forks source link

Add uwtools Python package #97

Closed christopherwharrop-noaa closed 1 month ago

christopherwharrop-noaa commented 2 months ago

NOTE: This is a sub-issue to #96 which encapsulates a large set of individual tasks.

A prerequisite to integrating with uwtools API for running UFS models in Chiltepin is the installation of the uwtools Python package. The uwtools Python package needs to installed into the container and also into the chiltepin Spack environment that is built on HPC systems.

To accomplish this, a pip install command needs to be added to the Dockerfile creation script and also to the install script. A pip install can only be accomplished by installing directly from the git repository with: pip install 'uwtools @ git+https://github.com/ufs-community/uwtools@main#subdirectory=src'

NOTE: The above example installs from main. To install a specific branch/tag, replace main with the name of the branch/tag. For example: pip install 'uwtools @ git+https://github.com/ufs-community/uwtools@2.2.0#subdirectory=src'

To verify that the install is successful, a test should be added that verifies correct operation of the uw command line and API. Perhaps simple calls to the CLI and API to get the uwtools version would suffice.

NaureenBharwaniNOAA commented 2 months ago

Question on this second part:

To install a specific branch/tag, replace main with the name of the branch/tag. For example: pip install 'uwtools @ git+https://github.com/ufs-community/uwtools@2.2.0#subdirectory=src'

This section is where I was going to add the pip install <uwtools> command to both of the script: https://github.com/NOAA-GSL/ExascaleWorkflowSandbox/blob/main/docker/spack-stack/create_dockerfile.sh#L76 https://github.com/NOAA-GSL/ExascaleWorkflowSandbox/blob/main/install/install.sh#L50

The command you mentioned here 'uwtools @ git+https://github.com/ufs-community/uwtools@2.2.0#subdirectory=src' I'm a bit confused on the structure. Does this import a package called uwtools via github and I think the main branch would be the best. We have have these directions as a user from uwtools documentation

christopherwharrop-noaa commented 2 months ago

The pip command installs uwtools from the Github repository. The name after uwtools@ specifies the branch or tag to install. The #subdirectory=src is needed to specify the path in the repository where setup.py is located so that pip knows what to do. I'm fine with using the latest/greatest uwtools, but the problem with using main is that it changes every time a PR is merged. So, the installs will not be consistent from time to time and we could end up with different versions of uwtools in different places. So, pinning to the latest release will ensure the same code is always installed, and we can just update the version as new uwtools releases are made. We can also point to main if we have to test something that isn't yet in a release.