Redesign bump-pom-scijava logic to be foolproof

[hinerm]

The required steps to bump and release new versions of pom-scijava, using the Bump-POM-SciJava script and corresponding bump-pom-scijava.sh, is somewhat convoluted and highly error prone when breaking changes are involved. Basically, we would like to move to a system where we can lean on the tools to help us decide what needs to be done (e.g. when there are breaking changes, what components need to be updated) and avoid ever needing to manually edit pom-scijava (e.g. to preemptively account for what versions of software will be released).

Thus a new overall workflow is proposed:

There will be a single bump-pom-scijava Jenkins job, with no parameters. It will perform the following operations:

• If there is a pom-scijava topic branch, check it out.
• Scan the complete software stack.
  • If there are new versions of any components, add commits increasing their version.property, and push to the topic branch.
• Run mvn install on pom-scijava. This will allow us to test each component against the new version properties without updating their pom-scijava version. NB: requires unified pom-scijava versions (which would be maintained by this script once established)
• for each scijava project that we have push rights to, starting at the lowest levels of dependency:
  • Checkout the latest master and attempt to build/run tests
  • If successful, continue
  • If unsuccessful, push a topic branch for that project equal to the current pom version, send notification of failure, abort bump-pom-scijava.
    • MANUAL STEP: fix the broken downstream project's topic branch to compile and test successfully against the pom-scijava topic branch. Merge to master and release the next version. Run bump-pom-scijava again.
• When all projects have passed:
  • Merge pom-scijava topic branch to master & delete the topic branch. Increase pom-scijava version and deploy. (optional: rebase and squish all the changes to a single commit)
  • for each scijava project that we have push rights to:
  • update to the newly created pom-scijava version and push to master

This will create a simple workflow where we just run bump-pom-scijava, respond to the feedback it gives us about failing projects, perform the necessary updates, and keep running bump-pom-scijava until everything passes. There is no ambiguity about what flags need to be selected in which circumstances, and no need to ever manually edit pom-scijava, or spend time trying to figure out what components need to be updated.

@dscho @ctrueden : any thoughts on feasibility, concerns, etc?

[ctrueden]

Thanks @hinerm, I think we need a process like this. It will be much less error-prone. It will remove the hardcoded list of SciJava-based projects from the bump-pom-scijava script. We can glean the relevant Git repositories from the SCM entry of each POM. We can also lean on Maven to do the dependency resolution, by synthesizing an aggregator POM at the top level. Then we only need to run mvn once at the very top level to test everything with the correct dependency order—assuming all projects have first been brought into line with the same pom-scijava version of course.

[ctrueden]

I completed an initial version of a script to scan the SciJava software stack: 8507f4b805fc855491d6fe6057426dab3b9ed3e3

[ctrueden]

/cc @joshmoore

[dscho]

How about making this part of a (to-be-created) scijava-maven-plugin?

[ctrueden]

@dscho: Yes, there are various bits of this work that would be perfect as part of such a plugin, as we discussed the other day.

[ctrueden]

The bump-pom-scijava script was retired earlier this year (e694e5eb0d230c973fe3690e9a36e9bf1cbc1fe8). We have the melting-pot to test for component version coherence. We have status.scijava.org to report which components are up to date, which could be incremented, etc. It is not always the case that pom-scijava should reflect the latest releases of things—the central point of a BOM is to list compatible versions, not necessarily latest versions, of components.

Since pom-scijava is the unified BOM these days, we are OK manually updating version properties as needed every so often.