Prototype release branch process

[adwk67]

Refinement

• done :heavy_check_mark: ~this issue can replace 235~
• done :heavy_check_mark: ~remove 235 and 236 from 208 and rename 208 such that it covers LTS-related tasks only~

Goal

Conduct a 23.01 release test-run using release branches:

• begin this early in the next release cycle to give enough time to iron out issues before the actual 23.01 release
• prototype a complete release i.e. all operators and docker-images

Acceptance criteria

• [x] a prototype release has been conducted multiple times (tests have been done for two docker image repos - zookeeper and airflow - and for one operator repo (zookeeper)
• [x] an issue has been opened for pre-requisite changes to all operator cargo versionshttps://github.com/stackabletech/issues/issues/314. N.B. the nightly versions will remain initially unchanged so that example etc. can still run against product images that actually exist (this may be refined later on)
• [x] a new release template issue (and an issue for 23.1 based on that template) has been created https://github.com/stackabletech/issues/issues/313
• [x] Superceded issue has been closed: https://github.com/stackabletech/stackable-utils/issues/33

How to test

From stackable-utils:

• run ./release/create-release-branch.sh -b 23.1 -c -p
  • this will create and push branches for the https://github.com/stackabletech/test-platform-release-zk and https://github.com/stackabletech/test-platform-release-images repos
• run ./release/create-release-tag.sh -t 23.1.0 -c -p
  • this will create and push tag 23.1.0 for the repos named in the previous point

Background

To consider

Discuss and document answers to the following questions:

• Stackable mono-repo, "yes", "no" or "not now"?
  • what happens to the product-specific READMEs?
  • what happens to templating generally?
• branch protection on the release branches?
• during the testing of release candidates, any fixes will made in main and cherry-picked to the release branches. Is there merit in doing it in the opposite direction?

Suggested/initial process outline

• create releases branches for the docker image repo and all operator repos
  • in the new branching system, we will maintain a release branch per platform release, i.e. a branch release-22.11, release-23.01 etc.. On these branches we will tag platform versions, i.e. 23.01.0-rc1, 23.01.0, 23.01.1.
  • (for the release itself - not this prototype - there is no need for a feature freeze)
  • As we now tag a whole platform release instead of individual products, the build system/GitHub actions should build all images from these tags.
• follow/adapt steps described in 262 (see also 264). A rough outline follows:
  • run release script to replace nightly everywhere with the platform release e.g. 23.01-rc1 and set antora docs to prerelease = true
  • see here for how cargo edit can help with this
  • also update operator versions in docs/templating_vars.yaml (can this be automated? difficult if not all operators are bumped)
  • when testing is complete, 23.01-rcXX will be replaced with 23.01 and set antora docs to prerelease = false
  • the release script can either receive these tags explicitly or contain logic that resolves the "next" appropriate tag
  • this will tag operator- and docker-image-branches with the same tag which should make documentation updates much easier as the tags are consistent everywhere
  • test the code
  • if 308 has not been implemented, run all getting-started scripts first to check (and, if necessary, fix) inter-operator compatibility
  • test all stacks/demos
  • test upgrade from previous release
  • update stackablectl/release.yaml to add the new release
  • test the documentation
  • examples should be runnable
  • antora tagging, drop-downs etc.
  • complete marketing tasks
  • short blog in EN/DE
    • involve marketing dept. so that is not purely a technical perspective
  • blog linked to from headline banner
  • video (optional)
  • announcement in github
  • create an updated release template for subsequent releases

[lfrancke]

Have you talked to/coordinated with @soenkeliebau and/or @sbernauer on this?

[adwk67]

No, mainly as the ticket is a consolidation of information in other tickets that we have also discussed elsewhere. We also mentioned (yesterday I think) that it doesn't make sense to refine this ticket 100%, as so much will/can only become apparent in the course of actually working on it (e.g. what we include in an updated release script). But I agree, it's important to at least discuss the two issues I mentioned at the top, that we can close/adapt: I'll make a note to bring it up during/after the daily tomorrow. We also have the mono-repo issue on the arch-board for tomorrow.

[lfrancke]

Sebastian, Sönke and I chattet last week about how images should be versioned etc. I'm fine with accepting this ticket but I want to make sure that it's coordinated with the two of them.

Will you do all the updates to the tickets above? If yes: I'm fine with moving this to ready now.

[adwk67]

Let's wait until we have sync-ed in/after tomorrow's daily, just in case I've missed any new developments - then I can update the tickets accordingly.

[lfrancke]

I'm fine with moving this ticket as soon as you think it's ready.

[adwk67]

Yes, I think it is refined as far as it can be.

[adwk67]

On-prem notes 28.11

• see photos below for details of flow
• remove rc part of tag
  • means we can use the same mechanism for the release process and later bugfixing
  • we thus start with e.g. 23.01.1
• only change pushed to main will be setting the version to nightly everywhere (rather than e.g. 0.7.0-nightly)

• 

[fhennig]

Thanks for putting in this effort!

With my docs hat on, I'm wondering what the process will be for the docs version (prerelease false/true)?

[maltesander]

In this approach we will basically only have a shared version for everything (docker images, operators). This will be something like 23.01. The main branch will always be nightly and prerelease=true, the created release branches (e.g. release-23.01) will be tagged with the release + fix like 23.01.0, 23.01.1 e.g. for each "fix" or backtrack and always have prerelease=false.

[sbernauer]

The docs stuff should be reflected in the diagram as well ;)

[adwk67]

From architecture meeting 07.12: