Open movermeyer opened 3 years ago
Update: I have created a new CircleCI orb that can call GitHub Actions with a repository_dispatch
event.
Update: @kylebarron was kind enough to open https://github.com/closeio/ciso8601/pull/109, which does the wheel build entirely using GitHub Actions. π π
For a couple of my repos, I have a process that's
bumpversion minor
or patch
or major
locally, which automatically bumps the version number in any required files, then creates a new git tag. https://pypi.org/project/bumpversion/git push --tags
which pushes the commit and tags to Github, and Github creates wheels and sdists and pushes them to PyPI: https://github.com/kylebarron/pymartini/blob/master/.github/workflows/build-wheels.ymlBut that process isn't currently set up to publish Github "releases" as well; this page is empty: https://github.com/kylebarron/pymartini/releases
@kylebarron
Thanks for the link to bumpversion
/bump2version
.
I was hoping to add a similar feature to changelog-cli
, but got distracted. π
bumpversion
is really simple so I'm a fan. I think now bumpversion
is just an alias to bump2version
, so I just stick with bumpversion
.
As it stands, there are a number of actions needed in order to release a new version of ciso8601
VERSION
insetup.py
CHANGELOG.md
We should be able to automate (nearly?) all of this.
Problems
Outline of a solution
We can use GitHub Actions to automate (nearly?) everything. Here's my current thinking of how it would work. I'm testing these configurations out on this test repo.
I imagine it being broken up into three GitHub Actions:
1. Create the draft GitHub Release
Trigger
A developer pushes a tag that matches our expected version number format:
v1.2.3
Results
2. Add the release artifacts to the draft GitHub Release
Trigger
The build of the wheels is complete in CircleCI, and a webhook is sent to GitHub.
This is not trivial.
CircleCI no longer has first-party support for sending webhooks on build completion. Instead, you are supposed to use a "notification orb" to do it. I found an orb that sends a webhook that looks like how the first-party webhooks from CircleCI used to look like.
GitHub expects a specific format for the webhooks that can trigger a
repository_dispatch
event.I'll likely fork the orb and publish a new orb that formats the webhook exactly how GitHub expects it to be.
Results
3. Publish the release artifacts to PyPI
Trigger
When the developer publishes the draft release in the GitHub UI.
Results
This can be done using the convenient
pypa/gh-action-pypi-publish
GitHub action, using GitHub repositorySecrets
. Secrets are encrypted values that GitHub can decrypt within the GitHub Action containers. That way, no one ever sees the decrypted values, and means that maintainers can be given the ability to publish to PyPI without needing the keys to the kingdom. Further, the secret would be a PyPI token that can be scoped to only work for this specific project.Resulting Workflow
This is what a developer would have to do after this automation was in place:
Notes and Q&A
GitHub Workflows can't trigger other workflows
GitHub Workflows can't trigger other workflows. But workflows can be triggered by the completion of other workflows (
workflow_run
).The important distinction is that there is no way (AFAIK) to pass the parameters from the upstream workflow to the downstream one.
Idempotency and error handling
There are many state changes described by this automation. Since these are not being done within any form of transaction, it will be important that each step is idempotent, so that the process can be simply restarted/retried without worrying about the state of the system after the failure.
Speaking of retries, there needs to be mechanisms to manually trigger retries in cases of failures. Neither GitHub nor CircleCI make this easy. In GitHub, you can use the
workflow_dispatch
trigger to manually trigger a workflow, though providing the same parameters that they workflow needs does not seem to be trivial. For example, I haven't yet figured out (I haven't put any effort into it yet either) how to dogithub.ref || github.event.inputs.ref
yet.In CircleCI, you can guard a workflow with a
when
clause and then the only way to trigger that workflow is via a POST to the API.Why not switch entirely off of CircleCI for GitHub Actions?
CircleCI allows for arbitrary Docker containers for your builds, while GitHub Actions only provides a few "blessed" runners.
Since we are hoping to use the manylinux docker containers as part of the build, we cannot switch entirely to GitHub Actions.