Release protocols and branch management

[martindholmes]

This is a general ticket where we can discuss the issues around release protocols (especially where hotfix point releases are concerned), as well as the storage or otherwise of things that seem more like products than source in the repo.

We have discussed the problem of storing the staticSearch.html documentation and the staticSearch.rng schema (as well as Schematron stuff) in the repo; while these are products, the source for them, staticSearch.odd, is also subject to exactly the same conflict issues as they are -- updates to documentation to match new features in development push the ODD file ahead, but a hotfix will require that it be rolled back and updated to include that hotfix info but not the still-unreleased feature development work.

[joeytakeda]

Posted a duplicate issue—just putting what I said there below:

As #220 has made obvious, we need to have some clear protocols in place to try and mitigate conflicts between the main, dev, and any feature branches. The most problematic, as far as I can tell, are conflicts that when there is some sort of hotfix needed to be made in the main branch, but, in the mean time, there have been changes to the ODD, since those changes proliferate through the RNG and the built HTML documentation that's stored in the repository. It would be made much simpler if those files weren't tracked by Git, but as @martindholmes points out, anybody using the repository at any point in time should have the most up-to-date and accurate schema and documentation.

So just thinking aloud about some possible approaches:

• We just live with this and deal with the conflicts when they happen and try to keep the cleanest set of changes possible

• One complicated and convoluted idea: We could introduce another branch--let's call it dev-actual—which is where we do all of our actual work to the code itself; this branch would NOT include the RNG or HTML. We could configure a GitHub action that, for every push to dev-actual, tries to merge dev-actual into dev, runs the tests, makes sure they pass, builds the RelaxNG and HTML, and then commits the RelaxNG and HTML, and then pushes that to dev. In the scenario where we have to do a hotfix in main, we could branch from main, fix the stuff and update the documentation for the release, re-release, squash all of the commits into a single commit, then cherry-pick that full commit back into dev-actual, which would then rebuild dev with all of the generated stuff.

• Another option: if we were to remove the schema and documentation from the repository, we could still make the most up-to-date version into a history-less branch--let's call it dev-docs—that's automatically generated and deployed on GitHub pages from every change to dev. That would mean the documentation and schema from dev is always available (and in fact, readable) and people wanting to use the latest version of the schema could point to the RNG in the dev-docs branch. The release would become a bit more complicated, though, since we'd then need to create zipped archives of all of the stuff that people need, including the HTML documentation and RelaxNG.

[joeytakeda]

Just coming back to this bit:

while these are products, the source for them, staticSearch.odd, is also subject to exactly the same conflict issues as they are -- updates to documentation to match new features in development push the ODD file ahead, but a hotfix will require that it be rolled back and updated to include that hotfix info but not the still-unreleased feature development work.

Wouldn't the same apply to every file in the repository? Unless I'm misunderstanding, if, say, a hotfix is required in StaticSearch.js, but there's already been development of that file, then we'll end up with the same sort of conflict.

From what I can tell, rebasing might our friend here:

https://git-scm.com/docs/git-rebase

From the docs:

Assume the following history exists and the current branch is "topic":

          A---B---C topic
         /
    D---E---F---G master

From this point, the result of either of the following commands:

git rebase master
git rebase master topic

would be:

                  A'--B'--C' topic
                 /
    D---E---F---G master

I think in the case of #220, replace topic with dev and master with main with commits E, F, G as the hotfix for 1.4.1

At that point, I believe we'd have a similar structure with dev and the feature branches, which would then need to be rebased similarly.

Of course, this is totally untested and I'm not even sure I'm reading it correctly, but, if I am, might be worth investigating

[martindholmes]

It almost certainly wouldn't work automatically. When I was resolving the conflicts in the ODD file, the way the conflicts had been combined by the initial merge resulted in broken XML because git can't know that a div (for example) must be moved as a block; it only looks at lines of code. There would be more chance of success with JS and CSS for sure, but the results would almost certainly be unpredictable. With the ODD file in particular, the What's New section gets updated differently in exactly the same location (its beginning) in the hotfix and in the feature branch; it's impossible for the rebase process to disentangle those changes and put them in the right order.

So I think there will always be a possibility that manual merging will be required once in a while, but we can minimize it by keeping our feature branches tightly focused on one thing, as well as short-lived, and doing all documentation updates in dev.

[joeytakeda]

Since this is a general discussion ticket (or at least it is for now), I'm going to move this to blue sky, but I imagine there will be other tickets that emerge from this one: we'll certainly want to write these rules down somewhere and there may also be some changes to the build process and release process etc.