search

ACCESS-NRI / access-ram-condaenv

A conda (mamba) python environment for running ACCESS-RAM
Apache License 2.0
0 stars 0 forks source link

Conda Environment Versioning #1

Open atteggiani opened 1 month ago

atteggiani commented 1 month ago

How do we want to handle the environment release versioning/naming?

The access-ram conda environment name is not to be related to any specific package version (different, for example, from the payu-condaenv that has multiple versions depending on its payu version) or python versions (for the moment).

How do you think would be best to name the conda package (and folders where the package reside on Gadi)?

I suggest for the name to simply reflect the release tag ( so something like access-ram/1.0 or access-ram/1.3, etc.)

Any strong opinions on this?

penguian commented 1 month ago

This could be the subject of a separate issue, but what will this environment contain? Consider that u-dg768 already uses hh5 conda modules.

atteggiani commented 1 month ago

This environment will contain any packages needed to run ACCESS-RAM that are currently loaded through hh5 conda. It is to replace the usage of:

module use /g/data/hh5/public/modules
module load conda

with this environment (location still to be decided but something like):

module use /g/data/<project>/public/modules
module load access-ram

In an approach similar to what has been done for ACCESS-OM2 and ACCESS-ESM1.5.

CodeGat commented 1 month ago

I suggest for the name to simply reflect the release tag ( so something like access-ram/1.0 or access-ram/1.3, etc.)

Will this be used for only ACCESS-RAM3 or for future versions(?) of ACCESS-RAM? If it's the former, maybe the environment should be access-ram3 instead of access-ram.

Furthermore, when you say the release tag, do you mean the tags in ACCESS-NRI/ACCESS-RAM3? In that case, we use a modified version of CalVer (YEAR.MONTH.MINOR eg. 2024.08.0) rather than the SemVer you have in your comment (1.0, 11.3),

atteggiani commented 1 month ago

Will this be used for only ACCESS-RAM3 or for future versions(?) of ACCESS-RAM? If it's the former, maybe the environment should be access-ram3 instead of access-ram.

That's a good question. Looking at the how to run instructions, ACCESS-OM2 doesn't seem to require a tailored environment to be loaded. It just uses the payu environment.

Does anyone know what is the plan for ACCESS-ESM1.5? @aidanheerdegen Since ACCESS-ESM1.5 will require some other modules (for example the um2nc-standalone), I believe it will need a dedicated environment (??).

I think it would be good to have an access-ram conda environments folder that might contain multiple folders depending on the model version (access-ram3, access-ram3.5, etc.) and each of these would have subfolders for the specific release version (for example access-ram3/2024.08.0).

One issue about this: With the scheme above a new environment (for example access-ram3/2024.08.0 would need to be released every time a new ACCESS-RAM3 model release occurs). However, a new ACCESS-RAM3 model release might involve no changes that would need an environment change (if for example the python scripts are not touched), meaning the environment wouldn't really need to be changed (previous environment release would still work fine). A solution to this could be to have symlinks instead of having a new environment folder, in case a previous environment is valid.

CodeGat commented 1 month ago

However, a new ACCESS-RAM3 model release might involve no changes that would need an environment change (if for example the python scripts are not touched), meaning the environment wouldn't really need to be changed (previous environment release would still work fine). A solution to this could be to have symlinks instead of having a new environment folder, in case a previous environment is valid.

We don't necessarily have to have them in lockstep. They can be versioned separately, and that seems to be the easier option.

atteggiani commented 1 month ago

True, then I would go with the SemVer for the access-ram environments release: access-ram3/1.0, access-ram3/1.1, access-ram3/2.0, etc.

jo-basevi commented 4 weeks ago

I think access-ram3 would clash with the eventual model modules names for loading spack executables - see https://github.com/ACCESS-NRI/ACCESS-RAM3/pull/2#issuecomment-2277203000 has module load access-ram3/pr2-6.

So might have to use access-ram-conda or something that is different?

CodeGat commented 4 weeks ago

Maybe access-ram-scripts?

CodeGat commented 4 weeks ago

I think @aidanheerdegen has suggested conda/access-ram

atteggiani commented 4 weeks ago

I think it makes sense to call it conda/access-ram to emulate the hh5 conda/analysis3 environments.

CodeGat commented 4 weeks ago

Sounds good to me

atteggiani commented 4 weeks ago

I would add the versioning in the name with a hiphen, as done in the hh5 conda/analysis3 environments (analysis3-22.01, analysis3-23.04).

So it would be something like conda/access-ram-1.0 or conda/access-ram-3.2, and conda/access-ram for the latest release.