Open harshula opened 2 months ago
Will be looking into this as well. We will need to verify that git checkout
s of spack
don't do anything funny to existing installations.
@harshula is the expectation that for some tag 0.21-2024.06.28
we would git checkout 0.21-2024.06.28
on the apps/spack/0.21/spack
? In terms of things that would break spack
mid-install, I think this would be the most likely candidate :D
I'd like to see the same functionality that is now available for spack-config
and spack-packages
repositories also made available for the spack
repository. We may not use it immediately.
My issue with this is that we could run into a situation where we have multiple models which all request to deploy to different commits within a single releases/v0.X
branch, which would lead to modification of spack
state during installs, similar to https://github.com/ACCESS-NRI/build-cd/issues/59.
As a compromise, could we keep the config/versions.json
as it stands now ("spack": "0.22"
, where 0.22
is the portion of the releases/v0.22
branch), but have a workflow that can change the commit/tag of spack
at a particular version (0.XX
) if nothing else is currently deploying to it. This can be done either based on modification to a config file, or as a user-dispatchable event.
Option one is to have a global build-cd
config item that is the tag/commit of spack for a certain releases/v0.X
branch to use. For example:
{
"spack-version": {
"0.20": "aabbccdd",
"0.21": "0.21-2024.07.09",
"0.22": "11122233"
}
}
This file would act as a versioned piece of provenance regarding what commit of spack
(per major version) is currently deployed. The workflow that changes the spack
version on Gadi
would be triggered by modification of this file.
In this solution, the workflow is an update to the existing Create Deployment Spack
workflow in build-cd
. If we give a different commit to an existing version of spack, it will update the version of spack on Gadi (if no other models are currently being deployed to it).
My issue with this is that we could run into a situation where we have multiple models which all request to deploy to different commits within a single releases/v0.X branch ...
Hi @CodeGat , How are you avoiding the same problem w.r.t. spack-config
and spack-packages
?
We haven't been able to, yet. But we shouldn't add any more possibilities for concurrent access violations.
Then a solution must also include a fix for the current way of using tags for spack-config
and spack-packages
. If not, the concerns are not describing a practical problem. If the git repository is only updated and never reset to an older commit, does that address your concerns?
Yes, that would. In that case, solution 1 seems to be the way to go
At the moment we appear to be able to clone
spack-config
andspack-packages
based on a tag. We don't appear to have that functionality when cloning Spack.We now have an ACCESS-NRI Spack fork where we backport essential forks from upstream's
develop
branch and/or create commits that we require. Hence, it is now important to be stricter with the state/version of the Spack repository that we deploy.