Branching Workflow for Release Action

[philnewm]

Is there an existing issue for this?

• [X] I have searched the existing issues.

Please describe the feature you have in mind and explain what the current shortcomings are?

I did some further research on #1 to figure out a strategy to group PRs/Features together to represent a release. In OpenPype Milestone were used to achieve this kind of effect, but for more abstract GitHub Actions a more generalized approach would be preferred.

I found a lot of different resources on how to handle releases in combination with branching strategies. Feel free to have a look in case you’re interested

• Release Strategies using Git Branching
• Git Branching Strategies
• Branching Strategy - keep it simple
• Release Branching Strategy Basics
• Successful Git Branching Model

After comparing different approaches to the former OpenPype approach, the general baseline of main, develop and features branches and the fact that each package may have explicit versions - it looks like a strategy around Release Branches would do the trick using the already existing structures with just minor adjustments.

How would you imagine the implementation of the feature?

Allow feature grouping for releases by using release branches. To implement a partially automated release strategy in GitHub Actions - Branches and Merge Strategies will need a clear Definition.

Permanent Branches - always available

• main
• develop

Temporary Branches - only active/needed under specific circumstances

• features
• release
• hotfix

While only main, develop and the current feature branches will be present most of the time - the others are defined as well.	branch name	main	develop	feature-branch	release	hotfix
Naming Convention	main	develop	<feature-name>	release-*	hotfix-*
Persistent	yes	yes	no	no	no
Branched off from	-	-	develop	develop	main
Merged back to	-	-	develop	develop + main	develop + main + (release)
Could be deleted after release	no	no	yes	yes	yes

In currently creating a flowchart like this including some adjustments and where GitHub Actions are supposed to kick in. But for now, check out the reference.

Most of these ideas are based on this article I linked earlier. Using dedicated release branches is not strictly necessary but can help to streamline the process a lot.

Other than that, the concepts of Pre-Releases, Release-Candidates and Releases would be created off the Release Branch following this strategy. Meanwhile, Nightly Build/Pure CI-testing runs could still be created off the develop branch. But since I haven't yet well define what one or the other should be, I'm not unsure here.

Are there any labels you wish to add?

• [X] I have added the relevant labels to the enhancement request.

Describe alternatives you've considered:

Additional options to collect PRs/Features for a Release Action

• Dedicated release issue (can only be created by maintainers)
• GitHub Milestone (as it was before)
• GitHub Projects (For the future)

I'll test these options and will decide on how to implement a default GitHub Actions behaviour based on the available data sources.

Additional context:

Let me know your thoughts on that one. If you see any potential issues using this approach, feel free to add them.

[iLLiCiTiT]

I don't think hotfix is a thing in addons (If we need a hotfix, we need to create release).

What I think might occur, is that we won't create release from main but from different branch e.g. when we need to add fix for previous minor version.

[dee-ynput]

Pipeline repos

I agree hotfix doesn't really have any value for us, and will even have less value when all the code will be dispatched in many repositories. Even release branches will give not much advanage when only a few persons at a time, if not only one, will be active on each repository: we will be not far from having a realease per merge. Also I don't think we want to support more than one release at a time. We're aiming at backward compatibility as much as possible precisely to avoid that 😛

I root for continuous delivery and zero release stress. So KISS feels critical to me. But I might still have a simplistic view...

I might be wrong but I think we don't need to find a better workflow but rather the most fluid automation of what we currently do: main/dev/PRs I'd says the friction revolves more around version bumping / changelog than branching. At least it is how it look from my far-from-code PoV :) It is a critical topic because with multi-repo the system will rely even more on semantic based dependencies.

Again: I might have a biased understanding. But "main, dev, Issus/PR branches, automated changelog from labels, release tags (with rc and +dev support)" sounds like a good setup to me. If we manage to have this with linting and unittesting safeguards, with the whole workflow explainable in 3 sentences, we're on top of the world ✨

Trust @mkolar, @iLLiCiTiT and @antirotor more than me on this though 😉

Server repos

The server is a different beast, but I don't see hotfix branches or release branches being a thing there either though. Even when the team will be doubled, that's max 3 persons at a time on a repos. The hotfix are released at light speed thank to docker deployment. Backend doesn't support several release at a time, and frontend is not sementic released (AFAICR).

@martastain and @Innders have been polishing their workflow lately, but I think there is still a lot of martin-magic behind it. Maybe there's some easy fluidity to gain there? 😉

NB: I don't want to overstep on anyone for the topic. I'm not sharing my appinion to make a point but because a dumb argument always help to trigger feedback from the experts 😄 Let's hear them now! 👂

[mkolar]

In general I agree with @dee-ynput here.

• Hotfix is not needed for pipeline. We've tried multiple times, and it always ended up being more hassle than worth. But keep in mind that hotfix branches are a concept that we could use at any point if need be anyways. They don't have to be formalised for them to be usefull.
• Release branches have value, but are mostly useful when building one larger project, rather than many small repos. The idea is that you lock a feature set for a release, then do a bunch of bugfixes and then release. This will hardly be the case for us. That all being said. Release branches do make a lot of sense as temporary short term state to allow for tweaking upcoming changelog, create proper version files and be able to validate everything before it goes out.

Say we want to release my-addon 1.1.0

• I trigger the release process
• CI created the release branch
• CI calculates the version number and adds to the right files in the release branch
• CI generates changelog and adds it into CHANGELOG.md in that branch and also into the description of a PR from Relese Branch to Main
• We check that it makes sense, tweak the pr description if needed with some human touch and approve for release
• We trigger the final release by merging release branch to Main.

After the merge:

• CI creates tag and release on GH
• CI action generates any builds, zips and artifacts we need and uploads them to github and market
• CI merges Main back to develop and deletes the release branch.

One thing to keep in mind, is that there are occasions where we want to keep back a PR from being released because it's a minor version bump (or major) that we're not quite ready for. Using release branches allow us to create a release/next-minor for that and stash then change in there until we're ready to release it, ergo, not blocking the rest of faster development.

Again, this is something we used to do what OP was smaller, but as it grew we weren't able to keep up with the cleanliness without proper CI and it got a bit fuzzy

[philnewm]

I'm finally back in and checked your ideas. @dee-ynput first: I'm happy to read any contribution/idea to this, since my workflow is mainly about:

1. Gathering information
2. Play around a bit and create a plan for a prototype
3. Propose the plan and gather feedback from others
4. Implement a prototype as proof of concept or whatever the first goal is

So I'm very happy to read your ideas and concerns 😃 And fortunately for this concept, release and hotfix branches would be handled the same from a CI perspective, so it doesn't matter if both or only one is used for a repo. And KISS is critical to me in this use case too, as I want to go for a scalable approach which enables smaller simple repositories to be easy to manage using CI Actions. And as soon as some repo grows bigger, the very same CI tools still work without any or just small adjustments. So necessary effort for maintainers in regard to CI tools should stay similar no matter how complex the repo grows. We will see If it works out this way, but this is at least my approach.

@mkolar the way you described the usage of temporary release-branches is exactly how I would imagine it. They should be CI driven and only require user interaction in places we want/need it. I can even imagine triggering Pre-Releases from a release branch to ensure it works as expected in case of custom adjustments or requirements. And you got that right, release branches enables continuous development and merging to development for upcoming features that are not related to the release branch (release in preparation). And since every release branch will on release (and merge to main) create a new tag on main, we can have full tracking on exact versions while not blocking any further development.

Just two more question regarding version numbers:

• Where do version numbers get stored (release tag | in a text file | somewhere else)
• How to calculate next version? (Bump-Labels from PRs | new version in text file | increase based on Bug/Feature-Labels)

Next steps

• Will implement a release action which works with only using development to main merge - so no release branch yet
• Implementation will feature a bunch of small Action module with specific steps to keep it modular
• This will hopefully show if there are already any special use cases which need closer attention or not
• Will give an overview of the data present by just collection of the available PR information
• A test how well the Labels work for the intended use-cases
• Afterwards further discussion on if and how to go about CI for release branches

[iLLiCiTiT]

So this usecase is not considered anymore? Release 1.1.0 is out, but we want to backport some bugfix for a client to older minor and create 1.0.10 .

[philnewm]

@iLLiCiTiT It looks like it's not considered atm, but a fair point IMO.

So far, I imagine the concept like this:

• Release versions increase their versions as they are supposed to
• Critical bugfixes for older versions can get implemented as "in-between"/bugfix release
• A Bugfix for an older version would just be a "bugfix release" which does not get its own tag on main, or should it?
• Will check if GitHub offers some feature to create such in-between bugfix releases for older versions
• Changes can get merged in the current develop branch/release branch if needed

This idea would align pretty well with the concept of release branches

How did you handle this so far?

[iLLiCiTiT]

Changes can get merged in the current develop branch/release branch if needed

It should be 2 separate PRs if should be ported to develop too. The bugfix in older version may happen in file which changed or was removed in newer version.

[antirotor]

One point about the release branches - to take into consideration is to support bugfixes while retain some frozen-in-time behaviour. One of the more requested long term feature is ability to freeze bundle per project:

Background: Longer running productions want to stay on the same set of (hopefully) proven tools without taking the risk of updating and hitting bugs in the middle of the production. With addon separation to their individual repository and independent versioning, we are moving to that goal.

My arguments for release branches there is to support addon versions living in long running productions. Say you have addon Foo in version 1.0.0 that is used in production running for one year. It is actively developed addon, so in few month you have version 1.5.0 that added bunch of features and settings. Then you find critical bug in fuctionality already provided in 1.0.0. Production is obviously reluctant to update to 1.5.1 because even if it fix the bug, it will maybe change the behavior they have set in the production.

With OpenPype and monolithic versions of AYON we sort of lost ambitions to backport fixes thanks to the code and functional complexity - it wasn't in our power to do so in the most cases. But now, we should reconsider. We saw what updating studios in the middle of production can do and even with diversification provided with the new addon architecture there will always be addons like houdini/maya/core that will be difficult to update without side effects.