Open ryankurte opened 3 years ago
signify could be one way to support that.
I have a Rust implementation of it. Though it's not been touched since 2018 it might be not too much work to update it & expose parts of it as a library so that binstall
could use it.
The Cargo.toml
would then have the public key listed and `.sig
file or similar contains the signature to verify.
So there are solutions like The Update Framework or Docker Content Trust or here on GitHub gpg keys. They are all kinda similar, they both have a client building and signing, a server holding the data and one the meta data. They are both not bound to a specific hoster, gpg keys are not only used on GitHub but for many git server distributions and docker has their protocol of image naming and the oci which can be implemented by any source. (Theres is the method for tls certs, a kind of trust chain but that’s not an option as currently there aren’t thousands of chainable entities, it would be the publisher of a crate and this tool/repo)
So how do I see this as an option for cargo-binstall
(Not a maintainer of developer of this projects, so just my 2 cents)
Pre-assumtions
How do I imagine this to work?
For this kind of flow you would have 4 parties:
cargo-binstall
In the end we would have two flows: publishing and installing.
How would publishing flow look like:
Cargo.toml
Points I’m not sure about:
How would a user flow look like:
cargo binstall
Things I’m not sure about the users flow:
What general limitations should be given:
Now I know I’m not a developer of this tool, but its a lovely one and this is just tickling the tech security part of my brain and one of the pain points of introducing this tool to other techies I got to know. And again, just my 2 cents (or at least some paragraphs of it 😅)
If someone is keen to have a discussion for effective measures or plans, I’m happy to help. In general, also happy to hear feedback and let some shoot bullets through my ideas (a german saying, hope it translates somewhat into english)
🦀
Some resources about encryption/decryption and secure software updates:
Thanks for the detailed exploration! I've reached for similar designs while thinking about this. However, I think:
Hence, my position would be that we need to have some third organisation to create and maintain such a design. Fortunately, there are things happening here!
Furthermore, for true adoption, my personal belief is that:
At which point, yes, binstall involvement becomes feasible.
So I just went full on rabbit hole style on this.
My conclusion is, the only potection given is that there was no tampering with the binary since the release.
Again assuming:
crates.io
will not be the only place of meta sourceBut if thats the option to go with, I will gladly have a shot at this and open a PR after my final exams (so in a month or so)
Just started to have a go at it and something just struck my brain.
A public key can't be modied if the meta host such as crates.io
doesn't let you overwrite the crate. But if I were to use the same key very time, binaries and sig files could be changed, what this verification would be here to prevent.
Well how does cargo handle something like this? the crates index contains a checksum which can be used to verify the file received from crates.io. It could work similar to this, the Cargo.toml
could have an array of checksums (as every file would have one) and after downloading the file the checksum could be calculated and compared to the ones in the metadata
this would mean cargo-binstall
is as secure as cargo install
A public key can't be modied if the meta host such as crates.io doesn't let you overwrite the crate. But if I were to use the same key very time, binaries and sig files could be changed, what this verification would be here to prevent.
the basic concept i was thinking about was less about tampering more how to tie a given binary from actions or S3 or whatever third party source was used for distribution to a given crate release, so you could be reasonably sure that the binary you have pulled was the one that was created in CI not something else uploaded elsewhere. obviously this still depends on the security of the keys in the CI environment, but i think it still an improvement.
while detecting re-publishing or changes is certainly interesting (as is trustworthy / verifiable CI) i think that has to fall outside of the scope of this project (at least until github provides a secure key store and trusted binary signing API 😂).
So then verifying a tls cert and enforce https would be enought to trust the source of the binaries?
Sigstore has reached v1.0.0 and it provides a rust crate sigstore-rs
for verifying the signature.
I don't know if this bit would be an issue...
The crate does not handle verification of attestations yet
but hopefully that's implemented when we get to this
For https://github.com/nextest-rs/nextest/issues/369, I had a look at https://docs.rs/sigstore/latest/sigstore/ and seems like it should be possible to:
cosign sign-blob
: https://docs.sigstore.dev/cosign/signing_with_blobs, including in GitHub ActionsNow, this just verifies that the bundle uploaded also matches the artifact uploaded. I think this would be a really useful initial step.
To make this better:
Cargo.toml
? This way crates.io becomes the root of trust). It's not totally clear to me how to do that from the current sigstore API. https://github.com/sigstore/sigstore-rs/issues/274#issuecomment-1607010911 mentions this, I believe:
I think the verify_blob* methods could use a variant that supports Fulcio based certificates + verification constraints.
I'm not a security expert and I'm almost certainly missing something. It would also be good to maybe open an issue with sigstore-rs people discussing this.
More thoughts.
I believe that GitHub Releases is not a reliable place to store artifacts that can never be changed in the future. Is this correct? Does uploading either the artifact or the signature bundle to OCI solve the "malicious user takes over GitHub Actions and changes old artifacts" threat model?
hmm, you can delete and re-upload artifacts, but so long as the signature is valid is it important that these are immutable? i think the risks one is attempting to mitigate are worth some though here.
slightly aside one of the things i've been thinking about a bit is how annoyingly repetitive setting up gh actions CI is for rust tools... maybe it'd be worth us investigating putting together a workflow template with variables for the usual stuff (rust version, platforms, platform packages, cross) that could include packaging / signing / publishing with whatever mechanisms we do use.
slightly aside one of the things i've been thinking about a bit is how annoyingly repetitive setting up gh actions CI is for rust tools
A couple of existing efforts:
hmm, you can delete and re-upload artifacts, but so long as the signature is valid is it important that these are immutable? i think the risks one is attempting to mitigate are worth some though here.
for folks worried about an exact binary they may need to pin a signature or a hash for a given platform and version
Yes, security-conscious users would like to pin the version and platform and have it be guaranteed to always resolve to the same artifacts. (modulo cargo-binstall itself being compromised, but they'll likely want to pin that to an exact version too)
most users will fetch new version / binaries anyways so long as they're correctly signed so is a new version meaningfully different from an updated binary with the same version? (eg. compromised actions / keys could be used to publish a patch release which any non-exact version filter would update to anyway)
Yes. I think publishing a new version that is bad is materially different from a malicious actor surreptitiously updating an old binary. (crates.io has the same philosophy, right?)
Ah sorry, NobodyXu doesn't work on upload-rust-binary-action. But it's part of the same general family of actions as https://github.com/taiki-e/install-action which they do work on, haha :)
I believe that GitHub Releases is not a reliable place to store artifacts that can never be changed in the future. Is this correct?
@sunshowers We could put the public key/checksum inside Cargo.toml
since it is actually immutable and you can count on it.
If the registry is hacked, then no matter how secure your GitHub release is, it won't matter since the attacker can change Cargo.toml
to point to whatever release they like.
Also, we are working on checksum support for registry #1183 , it will provide guarantees on security (presumably because crates.io index and the crates.io storage can be provided by two different sets of servers).
We could put the public key/checksum inside Cargo.toml since it is actually immutable and you can count on it.
Interesting idea -- how would you do that? I guess I imagined modifying Cargo.toml
as part of the release process was generally off limits.
We could put the public key/checksum inside Cargo.toml since it is actually immutable and you can count on it.
Interesting idea -- how would you do that? I guess I imagined modifying
Cargo.toml
as part of the release process was generally off limits.
You can put a public key inside package.binstall
under Cargo.toml
, then use that private key to sign your packages released.
Once the Cargo.toml
is uploaded to crates.io, it stays there and is immutable.
You could still setup automatic upload to crates.io on GHA using the recent crates.io scoped token (which we are already using) and it can be revoked if your GitHub get hacked somehow.
Gotcha! So, hmm, I think just storing the public key doesn't quite solve the threat model that I outlined, because: assume that the private key is stored as an environment secret on GHA. A malicious actor can:
In other words, we don't just need the public key to be stored in immutable storage, we also need some sort of identifier for the binary. This could be just as simple as a hash, or a certificate, or something that just makes sure that once a binary is published it never changes.
Also, users will have to manage their own keys, which is something experts can likely do but new users could have trouble with. (In this case, to perform a release via GHA, you'd have to store the private key as a secret, then carefully destroy the key material on the local machine.)
While asking people to use keys is definitely one way to go about it and does solve some problems, ideally users would also be able to perform signing via OpenID Connect.
read the private key from GHA
If you store that in the secret, then even the admin cannot read it, it can only be accessed inside GitHub Action. Although they can try to reveal it by changing the GitHub Action.
In other words, we don't just need the public key to be stored in immutable storage, we also need some sort of identifier for the binary. This could be just as simple as a hash, or a certificate, or something that just makes sure that once a binary is published it never changes.
Hmmm, perhaps we can store a root certificate in Cargo.toml
, then derive a public key based on hash of the crates tarball uploaded to crates.io?
(Of course, if you can generate a new hash/public-key and modify Cargo.toml
in each release, it will definitely guarantee security.)
While asking people to use keys is definitely one way to go about it and does solve some problems, ideally users would also be able to perform signing via OpenID Connect.
Thanks, I will read it later.
I think the hash of the crate tarball can definitely be used when verifying the pre-built binaries, after all, the binary is built from the crate tarball.
While asking people to use keys is definitely one way to go about it and does solve some problems, ideally users would also be able to perform signing via OpenID Connect.
I skim through it, honestly I don't think it will solve the "someone replace the release artifacts" given that it still requires the developer to provide an identity token either in CI if automated, or locally.
Perhaps I misunderstood and it does have mechanism to solve it, but IMHO the most secure way is still to provide a checksum inside Cargo.toml
and update it on every release, given that crates.io is immutable and always trusted.
Also cc @taiki-e since I would also love to hear feedback from you.
We can do away with user key management via the same process sigstore works. That is, in CI:
As crate publishes are immutable, new artifacts can't be uploaded for that version, and as the keypair only exists in the context of that one CI job, it can't be stolen in the future.
Verification is either:
or:
FYI, @sunshowers @passcod @ryankurte rust actually has a closed RFC, I think it's worth time reading.
https://github.com/rust-lang/rfcs/pull/3403
If the RFC is revived, then we definitely would like to follow suit here.
I think that's only for crates, though? not for binary artifacts
Oops, they propose adding sigstore to crates.io, so I thought it could be also used for verifying binaries.
Alright, how about this:
[metadata.binstall.signing]
algorithm = "minisign"
pubkey = "RWT+oj++Y0app3N4K+PLSYTKhtXimltIHxhoFgyWjxR/ZElCG0lDBDl5"
file = "{ url }.minisig"
We add support for this optional section to the binstall metadata. algorithm
and pubkey
are mandatory, file
is optional and defaults to { url }.sig
(where url
is the url we're downloading, and we include all the other fields too just in case you want to do something freaky).
algorithm
can initially only be "minisign"
because that's pretty popular, self-contained, doesn't have a thousand options, has good rust support, and keys are small. Later we can add GPG and Cosign and whatever else.
pubkey
is the string representation of the public key in whatever format is native to the algorithm
, in this case base64. Later we can add things like:
pubkey = { file = "signing.pub" }
pubkey = { url = "https://someserver.online/signing.pub" }
to support loading from elsewhere maybe.
Then binstall would, if the section is there:
file
template for syntax (to fail early if it's broken)file
templateWe should add:
--only-signed
flag to refuse to install non-signed packages--skip-signatures
flag to disable this entire thing (e.g. if a packager messes up their publish and someone really wants to download things anyway)That leaves the modalities of signing to the packager. As we've discussed, there's two main approaches:
--git
too.--git
unless the publish process commits the Cargo.toml with the signature.That will at least introduce package signing to the ecosystem; with this initial approach implemented we'll then be able to get more feedback from both packagers, users, and other community members. Because we'll namespace under metadata.binstall
, we won't step on any feet, and then hopefully in the long term a common solution will emerge (with cargo-dist, crates.io, or whomever).
How does that sound?
It sounds great and aligns with how I would like this to be done.
I've started work on a PR to do this!
Alright y'all, we've shipped the first version of binstall with signing support!
Find more info in the release notes and documentation.
cc @sunshowers @ryankurte @somehowchris @taiki-e
We will also open to new signing algorithms and PRs for improving our signing mechanism!
Tracking/discussion issues for additional algorithm:
It should be pretty easy to have a basic flow so CI created images can be signed, and verified on pull. Not sure what algorithm(s) / tools we should use? Accessibility is key, both IRL and in CI, and it needs to all work well cross-platform.
.sig
or appropriate to the format)