search

jazzband / help

Use this repo to get help from the roadies
https://jazzband.co/roadies
27 stars 7 forks source link

RFC: Adopt new security model for publishing #364

Open sethmlarson opened 6 months ago

sethmlarson commented 6 months ago

Proposal

Hey all! After speaking with @jezdez at PyCon US 2024 we discussed the current security model and we wanted to reimagine it using some new features that GitHub and PyPI provide. Specifically using GitHub Actions, protected by CODEOWNERS, gated on GitHub Environment reviews from team leads and then uploading directly to PyPI using Trusted Publishers.

Given recent history, there are valid concerns of malicious commits making their way into releases. My current thinking on this is to optimize for visibility of the content in GitHub (avoiding binary artifacts) and to have a plan for how team leads or admins can step in if something happens. We also want to formalize a process (and make it easy!) for team leads to review changes between a known good version and a new proposed release if that's not already happening uniformly.

Proposed adoption plan

Since this is a pretty substantive change we wouldn't want to roll this out without making sure it works as expected. My thinking is we'd copy an existing repository, make changes there, test it out, and then after we're satisfied move to a single real project, and then work on wider rollout (potentially with automation).

I am available to help figuring out the initial configuration (and perhaps would need additional privileges on this repository to do so) and can do some poking around with what projects are doing today to see how tough adopting Trusted Publishers might be. I can also help out with some of the manual aspects like adding Trusted Publishers and fanning out PRs to projects. Projects that have specific situations will need to provide some feedback on how their current processes can be accommodated.

For consistency and to facilitate future upgrades to the process I recommend standardizing on the names of release workflows definitions, GitHub teams and GitHub Environments (bikeshed values of: release.yml (project)-team-leads, and release).

Let me know if there are concerns with this proposal, I am happy to change it to fit project expectations!

### Tasks
- [ ] https://github.com/jazzband/help/issues/367
hugovk commented 6 months ago

I think this is an excellent idea!

For consistency and to facilitate future upgrades to the process I recommend standardizing on the names of release workflows definitions, GitHub teams and GitHub Environments (suggesting publish.yml (project)-team-leads, and publish are my bikeshed suggestions).

My empirically-backed bikeshed is release.yml:

sethmlarson commented 6 months ago

@hugovk Always good to have data, I've updated my proposal to match.

Mogost commented 6 months ago

I fully support this proposal. The reimagined security model using GitHub Actions and especially CODEOWNERS is a significant improvement that addresses current security concerns.

ncoghlan commented 6 months ago

The proposal makes sense to me.

I'd be happy to offer contextlib2 as a guinea pig (there are a couple of languishing requests to sync with a newer base version of contextlib, so it's definitely due to release a new version)

(Edit: I've had a productive couple of days, so the pending changes are ready to go. It's been 3 years since the last release though, so there's no problem waiting a few weeks to actually make the release. If the timeline is likely to be longer than that then I should probably just go ahead and release it the old way)

blaisep commented 5 months ago

I'm dropping this link here: https://github.com/hynek/build-and-inspect-python-package because it aligns with our direction and I'll eventually weave it into #366 or a derivation.

blaisep commented 5 months ago

(Edit: I've had a productive couple of days, so the pending changes are ready to go. It's been 3 years since the last release though, so there's no problem waiting a few weeks to actually make the release. If the timeline is likely to be longer than that then I should probably just go ahead and release it the old way)

Hi @ncoghlan , could we pair up? I'm interested in writing down the usual workflows to make it easier to onboard new roadies #366

btw, @sethmlarson , there is a lot of good work in here: https://github.com/hynek/build-and-inspect-python-package/tree/main/.github/workflows

We could take it further and include some services to could give actionable insight into the condition and progress of a project. Tools like these have useful reporting options that can build confidence in a project with little toil:

blaisep commented 5 months ago

@hugovk

My empirically-backed bikeshed is release.yml:

* https://mastodon.social/@hugovk/112054931871821050

maybe have a stage.yml or similar to push to the staging pypi (for the times we're not yet confident) ?

hugovk commented 5 months ago

maybe have a stage.yml or similar to push to the staging pypi (for the times we're not yet confident) ?

Yes, I usually push to TestPyPI for merges to main, and it works well in the same workflow, especially when using https://github.com/hynek/build-and-inspect-python-package.

For example:

blaisep commented 5 months ago

Another step which might please @sethmlarson is using hardened build images, like the wolfi-act eg. https://github.com/wolfi-dev/wolfi-act/tree/main/.github/workflows

webknjaz commented 5 months ago

My empirically-backed bikeshed is release.yml:

FWIW, I'm in the camp of ci-cd.yml because I believe in testing the artifacts before upload, not merely rebuilding them with no testing in between.

There's also a problem with GitHub, suggesting a poorly made starter workflow that @woodruffw is trying to fix (including in the GitHub's Docs, which is slightly better).

Though, my guide https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/ suggests a simplified version because I didn't want to introduce too many things. Still, I use the build->test->(pause)->publish->update-git structure wherever I can.

What @jezdez set up in this org already requires reviewing the dists by forcing uploads to Jazzband's own intermediate server, not to PyPI: https://jazzband.co/about/releases. That server is what gates the process and pushes the approved dists to PyPI post-review.

ncoghlan commented 5 months ago

Hi @ncoghlan , could we pair up? I'm interested in writing down the usual workflows to make it easier to onboard new roadies

@blaisep As far as I'm aware, https://jazzband.co/about/releases#continuous-integration covers the existing release process, so I don't think there would be a lot to see in a pairing session (final review to check repo contents are ready for the release, make tag locally, push tag to Github). The private repo sends me a link once the staged package is ready for review, and that's currently a manual task to actually pull it down and test it (hence this issue).

@sethmlarson If the release process intrinsically grabbed the new sdist and wheels and used https://diffoscope.org/ to compare them against the previous sdist and wheels, that would be pretty helpful. I'd started experimenting with that idea in a previous role, and found it could be incredibly helpful with ensuring unexpected changes didn't creep into maintenance releases.