Simplify RAPIDS branching model

[vyasr]

Problem Statement Currently RAPIDS uses a somewhat bespoke branching model adapted from GitFlow to manage development. Development for a given release occurs directly on a release branch (e.g. branch-24.02) until that release, at which point a new branch is cut for the subsequent release (branch-24.04) and new development occurs there. There is an intermediate time period (burn down) where both branches are actively under development. This approach is quite different from traditional Gitflow, in which development happens directly on a development branch (develop) until a release is near, which is when the release branch is cut and worked on. However, it does share a similar usage and naming of release branches (as well as hotfix branches).

As has been discussed in a number of venues -- 1, 2 3 (this one is the original article on Gitflow, which has been updated with the author's own numerous caveats upon seeing how widely Gitflow has been adopted and potentially misapplied) -- Gitflow has somewhat fallen out of favor due to its complexity and the high overhead of maintaining these branches. The amount of work and coordination required to manage development in this way is nontrivial and adds substantial friction. The RAPIDS branching model inherits these problems. These issues are particularly thorny because RAPIDS is in some sense like a distribution, in that we maintain a large number of separate repositories that need to be updated in lockstep and often have interdependencies that are based on the branches themselves (for instance, Github shared workflows, Docker images used in CI, etc). Reducing this overhead would be significantly beneficial to us. Any model that keeps the default branch constant over time would immediately help here.

Proposal RAPIDS should move to a simpler model. From some discussions offline, two good candidates are OneFlow and GitHub flow. Between the two, OneFlow seems like a slightly better fit since it does include some concept of feature and release branches, both of which we need, while Github flow seems a bit more tailored to projects that deploy continuously. In both cases we will need to make some modifications to accommodate our burn down period when two branches need to be managed live. If we go with OneFlow, then this is partly handled by the existence of a release branch. We would also need to decide whether to keep forward merges or to simply way until release to do a one-time merge of the release branch. Concretely, the two options I see are:

• Keep the forward merger bot available. During burn down we would cut a new release branch (say branch-24.06) and set up the forward merge to go from 24.06 to main, keeping main up-to-date. Then, when we are ready to release, tag the last commit on branch-24.06 and merge back to main.
• No more forward merges. Once the release branch is cut, changes for 24.06 will not manifest on main until the release happens, at which point we merge 24.06 into main and all changes are in.

The former requires a bit more automation maintenance, but I suspect that it will be worthwhile since we often have development early in the new release (24.08) that depends on changes only made during the last release's (24.06's) burn down.

[betatim]

it does include some concept of feature and release branches, both of which we need

What are the use-cases that need feature and/or release branches? Is there a commercial promise to provide fixes for past releases?

I'd argue that you only need a release branch if it is fairly common to apply fixes to past releases and make new release from that branch, after the next release has happened. Otherwise you can use main and tags. The justification that I most often hear for releases branches is "but we need a hotfix after a release!" but I'd argue you can just apply your fix to main and make a new release. The release branch only comes in handy if you want to apply a fix to an old release "a long time" after it was released.

I'd include some form of "fairly common" in evaluating this need because more branches is more complex than less branches. And if you only need the release branch every two years or so, it is probably worth not having it and biting the bullet of going back to an old tag and fixing things up "by hand".

I am mostly devil's advocating here, but I thin kit is worth it to try and start from the absolutely most minimal minimum (pack your bag for the expected, not the unexpected). Git and co have the tendency to make all of us lovers of complexity :D

[vyasr]

We need a release branch during burndown, as I documented in my comment above (see the "Proposal" section). It doesn't have to live on after the release, since we could cut hotfix branches by branching directly off the release tag, but it does need to persist throughout the burn down period.

Feature branches are nothing special. That's our default model for new development anyway. Supporting cases where a long-lived feature branch has many PRs made it to it before merging into main is fairly trivial and we already do that occasionally. I don't think that model would require any changes.

[jrhemstad]

I'd like to see this happen as well. CCCL has been using a fairly simply model that looks like GitFlow

• main is always the default branch and PRs always target main
• We create release branches branch/* when its time to start a release
• Fixes that need to go into release branches are made via PRs targeting main that are then backported to the release branch using GHA automation

[pxLi]

hi @raydouglass do you have any updates on this? The spark-rapids team plans to follow the same branching pattern. Please share any available details in advance so we can align accordingly. Thank you!