search

ACCESS-NRI / ACCESS-OM2

ACCESS-OM2: ACCESS Ocean-Sea Ice Model
Apache License 2.0
5 stars 0 forks source link

TEST: Prerelease Pipeline #43

Closed CodeGat closed 7 months ago

github-actions[bot] commented 8 months ago

This ACCESS-NRI/ACCESS-OM2 model will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

github-actions[bot] commented 8 months ago

This ACCESS-NRI/ACCESS-OM2 model will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

github-actions[bot] commented 8 months ago

This ACCESS-NRI/ACCESS-OM2 model will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

github-actions[bot] commented 8 months ago

This ACCESS-NRI/ACCESS-OM2 model will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

github-actions[bot] commented 8 months ago

This ACCESS-NRI/ACCESS-OM2 model will be deployed using:

If this is not what was expected, commit changes to config/versions.json.

CodeGat commented 8 months ago 
gitGraph
  commit tag: "2024.01.1"
  branch pre-2024.02.1
  commit tag: "2024.02.1"

And then when the next commit comes along:

gitGraph
  commit tag: "2024.01.1"
  branch pre-2024.02.1
  commit
  commit tag: "2024.02.1"

and then when merged:

gitGraph
  commit tag: "2024.01.1"
  branch pre-2024.02.1
  commit
  commit
  checkout main
  merge pre-2024.02.1 tag: "2024.02.1"
micaeljtoliveira commented 8 months ago

@CodeGat Looking at your diagrams, I've got a couple of questions:

  1. Are the pre-????.??.?? branches release branches? That is, branches where only changes directly related to the release are added (like updating version numbers, docs, etc)
  2. Is the actual release tag created from master or from the release branch?
CodeGat commented 8 months ago

Hey @micaeljtoliveira :)

  1. the pre-*.*.* branches are prerelease branches - they're special branches that can deploy to our prerelease environment on Gadi, to test that the changes that we are making to the spack.yaml and associated spack-{packages,config} repos will work when they are deployed for real. In our prerelease environment version of spack, these will have the spack env names <model>-<version>-<commit number> - for example, access-om2-2024_02_1-1. (Upon rereading your comment, you are correct. They are release branches :D )
  2. The actual release tag is created from the name of the prerelease branch - we just take off the pre-. Ex. pre-2024.02.1 branch -> 2024.02.1 tag on main.
micaeljtoliveira commented 8 months ago 
2. The actual release tag is created from the name of the prerelease branch - we just take off the `pre-`. Ex. `pre-2024.02.1` branch -> `2024.02.1` tag on `main`.

Sorry, I guess I was not very clear. What I mean was: which commit is tagged? The merge commit (created from merging the pre-release branch onto master) or the last commit in the pre-release branch?

aidanheerdegen commented 8 months ago

Does the tag have to exist on the GitHub repo? Could you just make the tag in the local git repo on gadi?

CodeGat commented 8 months ago

@micaeljtoliveira: It is at the HEAD of the pre-* branch, until the PR is merged, where it goes to the merge commit, to stay. We may need the tag because the style of the access-om2@git.<tag> spec in the spack.yaml needs to be linked to a commit to build.

@aidanheerdegen: no, it can't be local because spack goes to the remote to fetch stuff for a @git version.

CodeGat commented 8 months ago

Also feel free to unsubscribe from some of the notifications here, I'm using this PR as a test bed for the last bits of the workflow so it could get a little spammy :)

aidanheerdegen commented 8 months ago

no, it can't be local because spack goes to the remote to fetch stuff for a @git version.

Well then sliding tags it is I think. It is a very specific use-case.

Are tag protection rules worth looking into? Stop anyone using/abusing release and prerelease tags? I guess it doesn't stop someone from referring to them.

Put something in the docs to say these tags are unstable and should never be used to refer to commits?

CodeGat commented 8 months ago

They are only unstable when they are on the pre-* branch. Once they hit main they will not be moved. Does that change the wording, somewhat? And yes, we do have tag protection on *.*.* tags (https://github.com/ACCESS-NRI/ACCESS-OM2/settings/tag_protection)

The other option is to modify the spack.yaml for prerelease builds when it's sent to Gadi. We can changed the @git version to something like @git.pre-2024.02.1 and have pre-* tags be volatile. I don't like the idea of modifying the spack.yaml so it's different from what is deployed, even if it is just a tag name change. That logic also might be brittle to other models spack.yaml, but we haven't seen others, yet...

micaeljtoliveira commented 8 months ago

@CodeGat Another question: where do new developments go? To the main branch?

If I understand the discussion correctly, this "sliding tag" is used so that you don't have to edit the spack specs during the release process. This seems a bit unnecessarily complicated, as I don't see any reason preventing you to name the release branch the same as the final tag.

CodeGat commented 8 months ago

New releases go to the main branch, yes.

If I understand the discussion correctly, this "sliding tag" is used so that you don't have to edit the spack specs during the release process. This seems a bit unnecessarily complicated, as I don't see any reason preventing you to name the release branch the same as the final tag.

The sliding tag is used because the spack.yamls definition of the access-om2 spec is referencing a tag that doesn't yet exist (as we create a PR, and modify the spack.yamls access-om2 version to the one we want when it is merged). Regarding naming the (pre)release branch the same as the final tag, do you mean renaming pre-*.*.* to just *.*.*? That could be a possibility, I just added pre- as it is easier to write branch protection rules for it, and shows that it's not yet a release, but a PR towards one. But you're right, this does make it a bit easier! I'll think on it today.

CodeGat commented 8 months ago

Only issue potentially would be if the remote branch wasn't deleted after PR merge, wouldn't that mean that the tag and branch name would be the same, and that would fail?

micaeljtoliveira commented 8 months ago

Hmm. This is very confusing because you are sort of mimicking the git releases of OM2 (and other supported models, as I'm assuming you want to reuse this workflow for those too), but using your own workflow. I can see this getting very complicated and potentially becoming very hard to manage. It might even get worse if you also need to take into account changes to Spack itself, for example, when updating to a new Spack release.

Have you considered own this is going to work in practice when the model's git workflow is something like gitflow or [oneflowhttps://www.endoflineblog.com/oneflow-a-git-branching-model-and-workflow)?

Maybe we should meet so that I can draw some git DAG's for you to test exactly how your workflow would interact with the model developers' own workflow.

CodeGat commented 8 months ago

The core of the deployment logic (in access-nri/build-cd) should be fine for multiple styles of git workflows. The ci/cd logic in ACCESS-OM2 is specific to deployment/releases for that repository, only. Although it would be worth talking about standardization for git workflows for different models. That would make ci/cd workflows for other models that hook in to the build-cd workflows easier.

aidanheerdegen commented 7 months ago

It does feel like we might be talking at cross-purposes. Or at least I'm not understanding the problem you're seeing @micaeljtoliveira.

The main branch can be used just like a normal repo. Any workflow can be used for making changes to the main branch.

This workflow is just for very specifically named branches for deployment to gadi to allow for a pre-deployment step.