Investigate simplifying SPEC 8 through use of pypa/gh-action-pypi-publish v1.11.0+

[matthewfeickert]

As noted in https://github.com/pypa/gh-action-pypi-publish/discussions/281, in https://github.com/pypa/gh-action-pypi-publish/ v1.11.0

every project making use of Trusted Publishing will start producing and publishing digital attestations without having to do any modifications to how they use this action.

This is great news, so a big thanks to @webknjaz and @woodruffw for this!

For some of our packages that have upcoming releases we should investigate how the attestations differ from the actions/attest-build-provenance ones and what the verification workflow is like. If we like them, then we should revise SPEC 8 to just use these automatically generated attestations, simplifying the process.

[webknjaz]

@matthewfeickert so looking into SPEC 8, I noticed what I think is a misconception — the suggested GitHub Environment name uses a verb. This name correspond to the deployment target, a platform where dists end up, which would be PyPI, TestPyPI or some devpi etc. It's not a process but a place, which is why it's odd to see it starting with a verb.. So I recommend changing publish-package to just pypi. Also, if the version is known, I recommend adding url: https://pypi.org/project/<name>/${{ some.version.var }} next to that name — this displays that rendered URL in a couple of places, like the repository deployments page (example: https://github.com/cherrypy/cheroot/deployments/pypi — see how each deployment has a link to the corresponding version?) and on the workflow summary page, in the graph, right in the box representing the job.

Another thing is that the example there shows signing before the release happened. I think I saw it in some of @woodruffw's repos that this premature step might be problematic if publishing to PyPI fails and you may end up with published signatures for dists that would never appear in the official published lists. Imagine you have to trigger the release pipeline again keeping the version the same and end of with two sets of signatures for the same dist names. I would suspect the nonpublished ones could be used for something like dependency confusing attacks.

For that reason, I treat the event of successful publishing to PyPI the point of no return and do any signing after the fact, in separate jobs even. One of the tools, SLSA, even provides a reusable workflow that can't be embedded into an existing job as it's a bundle of jobs itself: https://github.com/ansible/awx-plugins/blob/c7fc0a1/.github/workflows/ci-cd.yml#L828-L1115 (demo workflow run graph: https://github.com/ansible/awx-plugins/actions/runs/10793404435).

[tupui]

Feel free to make PRs to suggest some edits 😃

[matthewfeickert]

Thanks as always for the rapid and detailed feedback, @webknjaz!

This name correspond to the deployment target, a platform where dists end up, which would be PyPI, TestPyPI or some devpi etc. It's not a process but a place, which is why it's odd to see it starting with a verb.. So I recommend changing publish-package to just pypi

Correct me if I'm wrong (though for my projects this works, so if I'm wrong then there's something rather unclear), but the name field is the name of the corresponding GitHub Actions deployment environment that has been created for the trusted publisher process. It can be arbitrary, and I'm fine with changing it, but I feel that this (that the name field isn't arbitrary but needs to match to a user created environment) is something not well explained in the existing pypa/gh-action-pypi-publish README docs. If I'm wrong, and you have done some extra nice magic that's great, but in my projects I've always needed these to match.

Also, if the version is known, I recommend adding url: https://pypi.org/project/<name>/${{ some.version.var }} next to that name — this displays that rendered URL in a couple of places, like the repository deployments page (example: https://github.com/cherrypy/cheroot/deployments/pypi — see how each deployment has a link to the corresponding version?) and on the workflow summary page, in the graph, right in the box representing the job.

Wow that's really nice! Also thanks for teaching me that the https://github.com/<org>/<project>/deployments/ page even existed!

Another thing is that the example there shows signing before the release happened. I think I saw it in some of @woodruffw's repos that this premature step might be problematic if publishing to PyPI fails and you may end up with published signatures for dists that would never appear in the official published lists

Yes, this issue of having multiple attestations exist with the same human readable tags has come up in practice before given reruns. Also AFAIK the attestations are immutable as

Public repositories that generate artifact attestations use the Sigstore Public Good Instance. A copy of the generated Sigstore bundle is stored with GitHub and is also written to an immutable transparency log that is publicly readable on the internet.

so I think they're up there forever. The motivation here for generating the attestation as close to the build as possible was to try to ensure that any subsequent steps between building and publishing (e.g. twine validation, uploading and downloading of build artifacts to GitHub between stages) couldn't introduce intermediate injection attacks on the build artifacts. Do you have thoughts on this? My guess is that you view the possibility of multiple attestations with the same human readable tags as a bigger security threat than trusting the other steps and services along the way to not be compromised?

[webknjaz]

Thanks as always for the rapid and detailed feedback, @webknjaz!

This name correspond to the deployment target, a platform where dists end up, which would be PyPI, TestPyPI or some devpi etc. It's not a process but a place, which is why it's odd to see it starting with a verb.. So I recommend changing publish-package to just pypi

Correct me if I'm wrong (though for my projects this works, so if I'm wrong then there's something rather unclear), but the name field is the name of the corresponding GitHub Actions deployment environment that has been created. It can be arbitrary, and I'm fine with changing it, but I feel that this (that the name field isn't arbitrary but needs to match to a user created environment) is something not well explained in the existing pypa/gh-action-pypi-publish README docs. If I'm wrong, and you have done some extra nice magic that's great, but in my projects I've always needed these to match.

That's right, this is exposed as a field in GHA workflow jobs. And when you configure trust on the PyPI side, it should be the same. If it's set, PyPI will additionally verify that it matches. So technically it's arbitrary and works for as long as it matches.

I'm talking about the underlying concept it corresponds to. It's something that I believe GitHub itself did a poor job explaining and naming. Within GitHub as a platform, there's this idea of deployments and automations can work with it through Deployments API: https://docs.github.com/en/rest/deployments/deployments?apiVersion=2022-11-28. GitHub Apps can publish deployment events into the platform, and other Apps can listen to those events and react to them. GitHub also records those deployments and the states reported within this framework. Those are displayed on the Deployments tab in repos (which usually only shows up in the UI when you use this feature). They can also show up in the pull request timeline (you might notice them when you have integrations like Heroku that deploy PRs for preview). And so the place where you deploy your code has a name. It is a target environment of your deployment process. It could be Heroku or an SSH-accessible service. Literally anything that you can conceptually declare a target — GitHub does not actually know what it is, just its name and the state + an optional URL.

When using GitHub Actions, though, you don't have to resort to making API calls. Instead, you specify that environment key. And that name you use there is this deployment environment. So essentially GitHub Environments is a different interface/frontend to the Deployments API. These environments get recorded/created on the GitHub platform on first use and then appear in your repository settings so you can apply various protections, or you can pre-create it in the repo settings manually. GitHub then integrates these deployment environments into GitHub Actions workflows to both let you represent systems where you're sending your thing and to enable you to have more control over allowing certain associated jobs to start or pausing before important reviewable job actions. This is another side of the integration. And if you use these environment names, they'll be enclosed in the OIDC-signed bundle of data that PyPI gets.

Also, if the version is known, I recommend adding url: https://pypi.org/project/<name>/${{ some.version.var }} next to that name — this displays that rendered URL in a couple of places, like the repository deployments page (example: cherrypy/cheroot/deployments/pypi — see how each deployment has a link to the corresponding version?) and on the workflow summary page, in the graph, right in the box representing the job.

Wow that's really nice! Also thanks for teaching me that the https://github.com/<org>/<project>/deployments/ page even existed!

You're welcome ;)

Another thing is that the example there shows signing before the release happened. I think I saw it in some of @woodruffw's repos that this premature step might be problematic if publishing to PyPI fails and you may end up with published signatures for dists that would never appear in the official published lists

Yes, this issue of having multiple attestations exist with the same human readable tags has come up in practice before given reruns. Also AFAIK the attestations are immutable as

Public repositories that generate artifact attestations use the Sigstore Public Good Instance. A copy of the generated Sigstore bundle is stored with GitHub and is also written to an immutable transparency log that is publicly readable on the internet.

so I think they're up there forever. The motivation here for generating the attestation as close to the build as possible was to try to ensure that any subsequent steps between building and publishing (e.g. twine validation, uploading and downloading of build artifacts to GitHub between stages) couldn't introduce intermediate injection attacks on the build artifacts. Do you have thoughts on this? My guess is that you view the possibility of multiple attestations with the same human readable tags as a bigger security threat than trusting the other steps and services along the way to not be compromised?

Yeah, there's also currently no association between the attestations and indexes they're intended for. So people are concerned about things like publishing a thing to TestPyPI and having it signed. And then, tricking the users into using those dists would be a possibility as they are trusted. The same goes for cancelled uploads. I think, there's a potential for rollback attacks here.

As for having the steps close to each other, it's probably not that big of a concern. This is happening within a pipeline that you control. If that pipeline is compromised, the timing wouldn't really matter, and this would mean more serious impersonation problems anyway.

Besides, with pypi-publish bundling the attestations, you'll already have one signature that's uploaded close in time. It should not be a problem if more signatures will appear later on, provided that verifiers would check several sources of provenance.