Minor updates to bugfix automation

[phargogh]

In recent discussions, we've decided to make a few minor changes to how release automation operates:

• [x] In addition to the other checks for whether release automation should trigger a release, the bugfix release automation should also check whether there are any user-facing changes to release, using HISTORY.rst as a proxy. If there are no user-facing changes, a release should not be triggered.
• [x] If the latest PR submitted is github-actions, it means that the PR last submitted was an automated PR and it should instead be assigned to $RELEASE_MANAGER. See example failed build: https://github.com/natcap/invest/pull/164/checks?check_run_id=743226968
  • [x] If we're using $RELEASE_MANAGER across multiple workflows, let's find a way to have that defined in a single place.
• [x] Figure out how to format HISTORY so that pandoc can convert correctly. The 3.8.4 release text was not formatted correctly when converted to Markdown. Related to #133
• [x] Use some new automation tag instead of including 'AUTO' in PR titles.
• [x] Make auto release program exit nicely (expected exit), but still send a slack alert with the verdict of whether to do a release
• [x] Although all of the steps for a bugfix release could be walked through by hand to do a manual release, it would be preferable if we could trigger the release automation on demand, either through a script or a web application of some sort to make the API call. The objective would be to put the team in control of decision making, not the automation.
• [x] Update the Release checklist wiki page with these changes.
• [x] 1 week updates is too frequent. We'll move this to every other week. There does not seem to be an obvious way to do this.

[phargogh]

Working on this in my fork on task/144-minor-updates-to-bugfix-automation

[phargogh]

Plan for improved release automation

Objective

The software team should be in charge of deciding whether and when to do a release. When we decide a release should take place, automation should be in place to make everything happen as expected.

Planned Pause Points

The automation workflow should have a few pause points, opportunities to decide whether a release should proceed.

1. Before the release process starts, we would decide whether a release should take place at all.
2. After creating the autorelease branch, release tag and committing changes from the first release script. This allows us to decide whether the changes should be pushed, where the changes should be pushed (e.g. fork vs upstream), or if additional changes need to be made.
3. After the repository has been tagged and binaries have been built, we should have an opportunity to decide whether to publish the release.

Automation

Initiate a Release

When we decide to initiate a release, the release manager will run a script locally to initiate the release process.

Assumptions:

• The ref that the local repo is checked out to will be used as the base for the autorelease/3.x.x branch that the script will create.

Invocation: ./scripts/release-1-initiate.sh <version>

What the script will do:

• Check that dependencies are installed (python for sure, but also any others needed)
• Check that the <version> tag does not already exist
• Update the User's Guide commit SHA in the Makefile to match the latest revision in the UG repo.
• Update HISTORY.rst for the release

Commit the changes

Once the script has finished, it'll be up to the user to commit these changes, but we can provide helpful git commands in ./scripts/release-2-commit.sh <version> to:

• Create a new branch off of ref, autorelease/<version>
• Commit these changes to Makefile, HISTORY.rst for the release.
• Tag the new commit
• Don't actually push, but provide a git command to push the new tag and branch to natcap/invest. We don't want to actually push because we might realize that actually there should have been other files committed, or we got the version wrong, or something like that.

When the changes are pushed, the push should then trigger a new build of the tagged revision. Artifacts will be published to GCS.

Publish a release

Assumptions:

• The release's builds have all completed as expected.

Invocation: ./scripts/release-3-publish.sh <version>

What the script will do:

• Check for dependencies (pandoc, twine, gsutil, hub for sure)
• Download needed binary artifacts from GCS.
• Create a release object on the natcap/invest repo based on the text of HISTORY.rst for this release
• Upload all relevant artifacts to the release
• Publish all python distributions to PyPI
• Create a PR from the autorelease/* branch used into master and assign it to the release manager.

Future possibilities

• [ ] Can this workflow be triggered within github actions? It might be easier to manage API keys that way (with the option of running everything by hand if needed) https://docs.github.com/en/rest/reference/actions#create-a-workflow-dispatch-event
• [ ] Can we bypass GCS and instead just use github's API to just access the build artifacts from github actions?

[phargogh]

I think we're all good to go on this at the moment!