Release tag mechanism for locked-down installs

[rafaelfolco]

Push a labeled version of crucible that passes ci after merging, including all pinned commits of the sub-projects. This release mechanism will be used for regression testing and also by users that need more determinsm from crucible stable installs that do not change over time.

Additional changes are required to implement the installation bits that will take advantage of this release tag mechanism.

[k-rister]

I think we might want to do some path/path-ignore stuff to allow you to work on this without having to run the entire CI stack every time.

[rafaelfolco]

I think we might want to do some path/path-ignore stuff to allow you to work on this without having to run the entire CI stack every time.

ack, will take a look

[k-rister]

Have you thought at all about using branches instead of tags, or at least being able to switch to a branch if needed? I can see us need to backport a hotfix potentially and that would require a branch...

[rafaelfolco]

Have you thought at all about using branches instead of tags, or at least being able to switch to a branch if needed? I can see us need to backport a hotfix potentially and that would require a branch...

That's an interesting idea. Maintaining multiple branches may be problematic in many ways... My suggestion would be only one single branch that points to the latest release, so we focus in one single backport. Say you have 2024.1, 2024.2, 2024.3, 2024.4, 2025.1, being 2025.1 the latest stable release. If you find a bug, so you'd only fix the latest stable branch, labeled as "stable".

[k-rister]

Have you thought at all about using branches instead of tags, or at least being able to switch to a branch if needed? I can see us need to backport a hotfix potentially and that would require a branch...

That's an interesting idea. Maintaining multiple branches may be problematic in many ways... My suggestion would be only one single branch that points to the latest release, so we focus in one single backport. Say you have 2024.1, 2024.2, 2024.3, 2024.4, 2025.1, being 2025.1 the latest stable release. If you find a bug, so you'd only fix the latest stable branch, labeled as "stable".

I don't think we can really limit it to a single releast unfortunately. If we have versions, and people are using those versions and they are experiencing a bug then we need to be able to fix it. This is kinda the whole reason that maintaining versions is something we have avoided...but if we are going to have versions then I think this is a necessary evil.

If we have versions 2024.1, 2024.2, 2024.3 and 2024.4 and we find a bug in code that has been around awhile then I think it's very realistic to think that we are going to need hotfixes to each version and we will end up with 2024.1.1, 2024.2.1, 2024.3.1, and 2024.4.1 :(

I suppose we could also consider that we should create our versions as 2024.1.1, 2024.2.1, 2024.3.1, and 2024.4.1 so that if/when we make hot fixes that the versions are logically contiguous with the updates being 2024.1.2, 2024.2.2, 2024.3.2, and 2024.4.2.

@atheurer @jtaleric thoughts?

[rafaelfolco]

Have you thought at all about using branches instead of tags, or at least being able to switch to a branch if needed? I can see us need to backport a hotfix potentially and that would require a branch...

That's an interesting idea. Maintaining multiple branches may be problematic in many ways... My suggestion would be only one single branch that points to the latest release, so we focus in one single backport. Say you have 2024.1, 2024.2, 2024.3, 2024.4, 2025.1, being 2025.1 the latest stable release. If you find a bug, so you'd only fix the latest stable branch, labeled as "stable".

I don't think we can really limit it to a single releast unfortunately. If we have versions, and people are using those versions and they are experiencing a bug then we need to be able to fix it. This is kinda the whole reason that maintaining versions is something we have avoided...but if we are going to have versions then I think this is a necessary evil.

If we have versions 2024.1, 2024.2, 2024.3 and 2024.4 and we find a bug in code that has been around awhile then I think it's very realistic to think that we are going to need hotfixes to each version and we will end up with 2024.1.1, 2024.2.1, 2024.3.1, and 2024.4.1 :(

I suppose we could also consider that we should create our versions as 2024.1.1, 2024.2.1, 2024.3.1, and 2024.4.1 so that if/when we make hot fixes that the versions are logically contiguous with the updates being 2024.1.2, 2024.2.2, 2024.3.2, and 2024.4.2.

@atheurer @jtaleric thoughts?

IMHO we should simplify this process as much as we can. If we find a bug, we can fix in the main branch and in the stable branch, which points to the latest stable release. If users really need a specific bug fixed in an old release, update or do your own backport.

[jtaleric]

Have you thought at all about using branches instead of tags, or at least being able to switch to a branch if needed? I can see us need to backport a hotfix potentially and that would require a branch...

That's an interesting idea. Maintaining multiple branches may be problematic in many ways... My suggestion would be only one single branch that points to the latest release, so we focus in one single backport. Say you have 2024.1, 2024.2, 2024.3, 2024.4, 2025.1, being 2025.1 the latest stable release. If you find a bug, so you'd only fix the latest stable branch, labeled as "stable".

I don't think we can really limit it to a single releast unfortunately. If we have versions, and people are using those versions and they are experiencing a bug then we need to be able to fix it. This is kinda the whole reason that maintaining versions is something we have avoided...but if we are going to have versions then I think this is a necessary evil. If we have versions 2024.1, 2024.2, 2024.3 and 2024.4 and we find a bug in code that has been around awhile then I think it's very realistic to think that we are going to need hotfixes to each version and we will end up with 2024.1.1, 2024.2.1, 2024.3.1, and 2024.4.1 :( I suppose we could also consider that we should create our versions as 2024.1.1, 2024.2.1, 2024.3.1, and 2024.4.1 so that if/when we make hot fixes that the versions are logically contiguous with the updates being 2024.1.2, 2024.2.2, 2024.3.2, and 2024.4.2. @atheurer @jtaleric thoughts?

IMHO we should simplify this process as much as we can. If we find a bug, we can fix in the main branch and in the stable branch, which points to the latest stable release. If users really need a specific bug fixed in an old release, update or do your own backport.

I have seen it both ways -- either branching or tags. I personally find tags easier to maintain, but that is due to my workflow.

Where I think things with Crucible get complicated are tagging across the project.. For example, does v0.0.1 (made up tag) of crucible work with v0.0.1 of rickshaw? Keeping the mapping clean might be problematic -- maybe I am overthinking it?

[k-rister]

Have you thought at all about using branches instead of tags, or at least being able to switch to a branch if needed? I can see us need to backport a hotfix potentially and that would require a branch...

That's an interesting idea. Maintaining multiple branches may be problematic in many ways... My suggestion would be only one single branch that points to the latest release, so we focus in one single backport. Say you have 2024.1, 2024.2, 2024.3, 2024.4, 2025.1, being 2025.1 the latest stable release. If you find a bug, so you'd only fix the latest stable branch, labeled as "stable".

I don't think we can really limit it to a single releast unfortunately. If we have versions, and people are using those versions and they are experiencing a bug then we need to be able to fix it. This is kinda the whole reason that maintaining versions is something we have avoided...but if we are going to have versions then I think this is a necessary evil. If we have versions 2024.1, 2024.2, 2024.3 and 2024.4 and we find a bug in code that has been around awhile then I think it's very realistic to think that we are going to need hotfixes to each version and we will end up with 2024.1.1, 2024.2.1, 2024.3.1, and 2024.4.1 :( I suppose we could also consider that we should create our versions as 2024.1.1, 2024.2.1, 2024.3.1, and 2024.4.1 so that if/when we make hot fixes that the versions are logically contiguous with the updates being 2024.1.2, 2024.2.2, 2024.3.2, and 2024.4.2. @atheurer @jtaleric thoughts?

IMHO we should simplify this process as much as we can. If we find a bug, we can fix in the main branch and in the stable branch, which points to the latest stable release. If users really need a specific bug fixed in an old release, update or do your own backport.

I have seen it both ways -- either branching or tags. I personally find tags easier to maintain, but that is due to my workflow.

Where I think things with Crucible get complicated are tagging across the project.. For example, does v0.0.1 (made up tag) of crucible work with v0.0.1 of rickshaw? Keeping the mapping clean might be problematic -- maybe I am overthinking it?

@jtaleric I think we have the cross project part figured out. The real question for you is what are your thoughts on backported hotfixes -- if in Novermber/December you are running Crucible version 2024.3 (ie. our 3rd quarter release) and we discover something is broken that you need a fix for, do you want to be forced to upgrade to the current release's hotfix (2024.4.x) or do you want that hotfix backported to the release you are on so that you can transition from 2024.3.x to 2024.3.x+1. Based on the conversations we have had I would think you want 2024.3.x+1, but maybe I'm wrong.

My current line of thinking is that we would "support" the last year's worth of releases -- so basically there would always be 4 supported versions available since we are currently planning a quarterly release schedule. The only backports would be for critical hotfixes -- if a user wants new/added functionality they either need to upgrade to one of the newer releases or move to the bleeding edge which is HEAD of master/main (ie. what everyone is running right now -- assuming they update in a timely fashion). The question is would the hotfixes be applied to all 4 releases or just the most recent one. I'm of the opinion that it doesn't make sense to "support" releases if we aren't going to fully support them with backported hotfixes.

FWIW, I sorta came up with the year of "support" when I put together the prebuilt image stuff. As currently configured all the prebuilt images are valid for at least a year, if not longer (they will be valid as long as they are the most recent version of the image and/or they are actively being used with the expiration timestamp refresh stuff enabled).

[k-rister]

Go ahead and drop the paths-ignore stuff and then I'll approve it. I don't want that interfering with anything as you start testing stuff.