search

ACCESS-NRI / spack-config

Shared spack configuration
Apache License 2.0
1 stars 0 forks source link

Add command to "clone" *Spack* environment that was used to build executables for a specific model config release #43

Open atteggiani opened 1 month ago

atteggiani commented 1 month ago

Description

Currently there is no "easy" way for a user to "clone" a Spack environment that was used to build the executables used in a specific released model config (for example ACCESS-ESM1.5-1-historical+contentrations-1.1).

Why it matters?

Having such a command would make it easier for a user that wants to change the source code of a specific model component (for example mom5) to have, as a "base" Spack development environment, the exact one that was used to build all the configuration executables (ACCESS-ESM1.5-1-historical+contentrations-1.1 in the example above), before modifying the Spack environment, changing the desired package and building it within the "modified" Spack environment (using spack install ...).

This keeps track of provenance and ensures all the packages in the "custom build", with the exception of the modified package (mom5 in the example above), are exactly the same as those in the released "base" configuration (ACCESS-ESM1.5-1-historical+contentrations-1.1 in the example above).

Proposed solution

I propose to create a command that checks the model name / tag of a released model config (for example from the modules.load field in the config.yaml file (for example, the tag for the file in the example above would be access-esm1p5/2024.05.1) and creates an environment (through spack env create -d <env_directory> <spack.lock_file>) using the spack.lock file inside the ACCESS-NRI release Spack environments folders on Gadi (/g/data/vk83/apps/spack/*/spack/var/spack/environments), from the environment that matches the model config name / tag (/g/data/vk83/apps/spack/0.22/spack/var/spack/environments/access-esm1p5-2024_05_1/spack.lock for the example above).

Practical Example

I ran ACCESS-ESM1.5-historical+concentrations-1.0 and now I want to run the same configuration with a modified cice.

I use the command (for now I will call it spackenvclone to make it clear, but the naming can be discussed) to clone the environment:

spackenvclone [-d <my_directory_for_the_environment>] ACCESS-ESM1.5-historical+concentrations-1.0

and I have the "base" Spack environment to start with, as a copy of ACCESS-ESM1.5/2024.05.0 (their spack.lock and spack.yaml files will be the same.

We can also add the option to omit the tag and get the latest environment:

spackenvclone [-d <my_directory_for_the_environment>] ACCESS-ESM1.5-historical+concentrations

would "clone" the ACCESS-ESM1.5/2024.05.1 Spack environment (latest at the time of writing).

Notes: access-om2-configs/#142 will allow to have the same conceptual command for ACCESS-OM2 and ACCESS-ESM1.5.

atteggiani commented 1 month ago

@jo-basevi @harshula

harshula commented 1 month ago

Hi @atteggiani , Thanks! This should make the user experience better.