Improvements to the release process

[wagoodman]

Today we have a release process that is relatively simple: push a tag, a team member needs to approve, the pipeline runs, and there is a draft release ready for someone to manually publish. In about 20 minutes you could go from "need to release" to "release is published" state. That being said, there are still some frictions here, I'll highlight a few that come to mind for me:

• Since we need to run on a mac for the signing step and we need docker for other steps in the the build process, we end up needing to install docker on a mac. This is about 10 - 13 minutes --half of the release process. Tracked in https://github.com/anchore/syft/issues/577
• ~The current changelog generator takes a few minutes to run, where the set of API calls to GitHub seems to be the limiting factor here. We have a relatively small repo and am a little surprised at the running time.~ Chronicle has been integrated to alleviate this problem
• We have a GPG signing step which we never finished (never published the public key), though the potential value for finishing this is high from a security perspective, key management and long term support doesn't seem appealing. There are probably better ways to do this today.

Some of these frictions hint at changes we can make, but this also good time to look for opportunities to improve the release process as a whole.

Feel free to add on frictions you've noticed or opportunities we should consider.

[spiffcs]

It also looks like we can update how we're using the seucrity scan action:

--
1 issue was detected with this workflow: git checkout HEAD^2 is no longer necessary. 
Please remove this step as Code Scanning recommends analyzing the merge commit for best results.
 

[luhring]

This is a great issue. Adding a few frictions I've seen:

• Tagging a commit before we're certain that we're releasing from that commit can cause issues for consumers of our code. Anecdotally, several folks seem to trust that a release tag won't change, once applied. This expectation is intensified in the Go world, because the first time anyone consumes our Go module at the tag, that tag is bound to a digest of our module in the public Go proxy, such that if we re-tag later, users trying to consume the updated tag will see errors from the Go toolchain. Since there exists a perceived "finality" to release tags, it'd be great to find a way not to tag until the release is final.
• We publish updated Docker images and homebrew formulae for releases before the releases are confirmed. If we look at a release draft and decide to substantially change or cancel the release, a burden is placed on the release operators to remember to unpublish the Docker images and homebrew formulae.

[wagoodman]

concept branch for part of a solution https://github.com/anchore/syft/compare/refactor-release (note, this branches from the install.sh work done previously, so needs to be rebased/cleaned-up when the install.sh PR lands)

This splits up the signing and notarization, and enables a local signing workflow that can optionally be done with snapshot builds. No additional artifacts are created from the signing step (no dmg or zip with a notarization ticket included).

[jonasagx]

Great issue. Adding an item to the list:

• Document the manual publishing step.

[jonasagx]

From refinement. Possible approaches:

• It's better to release maybe broken release notes, than have brew be broken while waiting for release to go from draft to published, we can always go back and fix notes.

Update: we're doing at least this so that the release cannot be in a broken state for this reason again (easily):

• https://github.com/anchore/syft/pull/1016
• https://github.com/anchore/grype/pull/766

[wagoodman]

Good final state (summarizing from refinement conversation):

• continually create a draft release from changes on the tip of main
• draft releases would speculate the release version based on the set of merged issues/PRs
• break out docker image/manifest, brew crafting and uploading, and mac notarization to a POST release process
• the "release" process would consist of only assets that are uploaded to the github releases page
• git tag would occur once the draft release is published as a final release (tag driven from the release tag)
• use chronicle (or goreleaser feature?) to speculate the version baked into the binary / used as the post-release git tag / used as the release title in release notes
• should be able to override the version in the draft-release workflow manually via workflow_dispatch (in cases where chronicle cannot speculate the version correctly.... the binary assets must have the corrected string)

When someone wants to cut a release... They would head into the github releases page and find the current open draft release (there should always be one that is continually updated... just like the github actions repos we have today). They would update the release notes making any manual changes they would like before clicking "publish". They would also be responsible for validating and correcting the version.

What impacts does this have?

• PRO: post release actions can be continually run on failure / intermittent connections (docker push, notarization issues, etc)
• PRO: today we install docker as a first step to release, THIS CAN BE REMOVED 🎉
• CON: since signing today requires running on a mac tip-of-main draft releases (which now would run more often) would be running on more costly runners
  • we may be able to get around this with extra tooling (and run signing on a linux box)
• CON: not all artifacts would be controlled under goreleaser (not a huge con)... this means we need to manage these processes ourselves.

Questions:

• goreleaser has the ability to append release notes to draft PRs, can we use this?
• can we use goreleaser to push the tag after-the-fact?

[wagoodman]

In between solution: Add a make target that facilitates cutting a release for someone. It would:

• run chronicle in advance
  • showing the changelog + all misc. changes that are not associated with a label (helpful for manual fixing of labels + changelog)
  • speculate the release version based off of the changes
• prompt someone for the release version, with a default value of the chronicle output
• hitting enter will tag and push the release tag...
• ...and open the github actions url to the run (bonus)

This is not nearly as much work as the final state but is an improvement for some of the frictions today.