RFC: Proposal for new signing mechanism

[rochlefebvre]

Rendered RFC.

We'd like to propose a new gem signing mechanism. We believe the new mechanism will be easier to use and more secure. Our goal is that eventually, almost all gems are signed as a matter of course. The proposed design builds on sigstore, an OpenSSF-backed project for software signatures.

As part of our work, we have developed a proof of concept system that works as a gem plugin.

We have drafted the RFC for an audience who are familiar with RubyGems, but who may not be familiar with sigstore or with the security primitives it utilizes. We note that the reference-level guide section is very detailed, and most of it can be safely skipped on a first reading.

We will be available on the Bundler Slack if you wish to discuss there.

Contributors: @bettymakes, @aellispierce, @jchestershopify, @jenshenny, @tomstuart, @doodzik and @rochlefebvre.

Special thanks to: @dlorenc, @lukehinds and @bobcallaway for help on sigstore questions.

[jchestershopify]

Have any of the rubygems maintainer team been on this discussion yet?

We've seen @simi so far, but there are others who haven't weighed in. I'm keen to hear from @sonalkr132, who does a lot of the operational work on rubygems.org. The rubygems-as-IdP proposal would affect rubygems.org more than the original proposal.

Including them and their perspective and concerns and interests in this discussion up front seems important.

Agreed and we are eager to listen.

[haydentherapper]

As discussed above, maintenance of an IDP is not trivial. This will require significant knowledge of OIDC/OAuth, along with the operational costs and security risks associated with running an IDP.

I would recommend focusing on implementing signing support initially using public IDPs, and revisiting running a RubyGems IDP during a later iteration.

For those who are not comfortable with including their email address in the issued certificate, it's worth noting that Sigstore supports Github Action OIDC identities, as noted in https://github.com/rubygems/rfcs/pull/37#issuecomment-1029443906. Signing that is initiated using a Github Action will only include information about the repository in the issued certificate. Github Actions is also beneficial for those who want automation, without a browser.

How do we get the gem owner’s profile URI into the signing certificate’s subject alternative name? What claim or scope do we use? How do we standardize with other package managers?

Adding onto https://github.com/rubygems/rfcs/pull/37#issuecomment-1029443906, we will likely need to make a few changes to Fulcio to support whatever is decided. The subject of the token could either be the profile URI as a URI Subject Alt Name or the username of the Gem owner (Fulcio could append a configured domain to construct a SAN URI, such as URI:username@rubygems.org)

[raggi]

Could you include both guide level documentation and technical documentation for a revocation model? Many supply chain failures with libraries involve unchecked patches or developer machine exploits that produce or enable signed package productions that would be best later revoked. What will that flow look like?

[rochlefebvre]

Could you include both guide level documentation and technical documentation for a revocation model? Many supply chain failures with libraries involve unchecked patches or developer machine exploits that produce or enable signed package productions that would be best later revoked. What will that flow look like?

Good question! We briefly and indirectly touch on that in the unanswered questions.

We envision that the immutable transparency log would contain more than gem maintainer signatures. 3rd party reviews, gem yanks, changes to a package's owners, etc. A gem client would query the log using the gem digest (or perhaps using other fields, TBD) and would compute its effective state. If a seemingly properly signed gem is later revoked by rubygems.org or flagged by a credible entity, then the gem client stops the installation process.

These sorts of attestations and gem installation policies build on top of what we are proposing in this first RFC. They are currently on the cutting floor so we can pitch a more concise approach to gem signing.

[rochlefebvre]

Here are some early thoughts and clarifications on Jacques' Do everything with Rubygems.org IdP scenario. I'll state the obvious here: these are relatively new ideas, subject to change.

Rubygems.org is the only authority on its profiles. If we intend to have "https://rubygems.org/profiles/flavorjones" as the signing subject instead of paul.b.flavorson@example.org, then sigstore needs to receive a bulletproof attestation that the requestor can indeed access that rubygems account.

In the RFC's detailed signing flow, steps 1-10 show the gem client making calls to oauth2.sigstore.dev, and not to GitHub directly, save for a few sign-in redirects. sigstore leverages Dex, an identity provider that presents a common OpenID Connect interface to sigstore clients (like gem), and supports many identity providers over various protocols.

sigstore's instance of Dex ~currently supports GitHub, Microsoft, and Google~ (edit: not all providers go through Dex, it turns out. I don't know which ones do). I believe that Dex could be configured to interface with rubygems.org using an OIDC connector. For the purposes of code signing, Rubygems.org does not need to implement all of OAuth2 & OIDC, and it does not need to support client registration. sigstore can be the only supported client. The new identity provider only needs to meet Dex's technical prerequisites, along with the sigstore group's criteria for identity provider security & oversight.

Requirements are TBD as of this writing. I can think of a few:

• OAuth2 authorization code flow
• support for single use refresh tokens
• fairly barebones support for OpenID Connect:
  • a well-known configuration endpoint
  • /auth, /token, and /jwks_uri endpoints
  • no need for custom scopes or claims

At the risk of solutioning too early, I looked at doorkeeper, a gem that adds OAuth support for Rails apps. It's mature, supports many standards, has a lot of contributors, and a permissive license. Here's a rubygems.org PR that takes it out for a spin. I believe that doorkeeper experts and sigstore/Dex experts could hash out a minimal configuration that satisfies sigstore requirements.

[rochlefebvre]

Update on using a rubygems.org profile URI as the signing subject.

As a precondition for supporting a new identity provider, the sigstore maintainers require that the subject identifier (e.g. the profile URI) must not be reassignable to another entity. If https://rubygems.org/profiles/qrush represents User 1 today, it cannot resolve to User 2 in the future.

This is incompatible with our preliminary ideas for using a profile URI having a user handle in the path, because user handles are not permanent.

I can think of two ways to reassign a user handle to someone else:

• User 1 edits their profile by choosing a new handle. Any user may then choose the free handle.
• User 1 deletes their profile permanently. Any user may then choose the free handle.

Handles are case-sensitive. Foo and foo are different users.

Profiles may also be referenced by their user record ids. https://rubygems.org/profiles/qrush is also known as https://rubygems.org/profiles/1. The profiles api endpoint also allows either addressing scheme, and returns both:

https://rubygems.org/api/v1/profiles/qrush.json
{"id":1,"handle":"qrush"}

id-based profile URIs are stable, and may not be reassigned nor reissued as part of another user creation. If the profile is deleted, then the URI points to nothing. This too could change if the repo supported some kind of user soft deletion (or just copied the deleted user into a deleted_users table for posthumous lookup).

For these reasons, I recommend that we use the user id version of the profile URI as the signing certificate's subject alternative name. The handle could also be recorded in a custom extension, with the understanding that it is not authoritative beyond the time of signing.

[rochlefebvre]

Hi @simi! Early on in the RFC review process, you expressed interest in finding alternatives to relying on 3rd party identity providers as the primary way to identify gem signers. You entertained the idea of having rubygems.org as an identity provider.

I'm all for exploring this avenue; it avoids a few problems from the 3rd party IdP/email approach. I have done some preliminary research, and I have started figuring out what this approach entails for both rubygems.org and for sigstore.

Before people put too much time researching the option, I'd like your take on the intended result. What do you think of this alternate proposal?

[rochlefebvre]

A few weeks ago, Jacques proposed an alternative approach to signing gems using an identity provider-verified email address. Instead, gems would be signed using a maintainer's RubyGems profile. As a reminder, here are some pros and cons to this approach:

Do everything with Rubygems IdP

Pros No email addresses need to be public (rubygems.org account becomes the signing subject) Every maintainer is already on rubygems.org No “vendorizing” No need for an “email option” Arguably, signatures from a new rubygems.org owner is a stronger signal than signatures from a new email address

Cons No available proof of concept Will take longer to implement Effort and cost to operate the new functionality A new single point of failure - rubygems.org account takeover enables fraudulent signatures

To me, the killer feature is the clear identity of the signer in the signing certificate: the rubygems profile itself. That got me thinking about other public identities that gem maintainers may wish to sign their work as. Maybe a GitHub or Twitter profile?

A GitHub profile-based signature makes sense to me. It has the same pros as an RubyGems IdP, minus the vendorizing. It also sidesteps most of the cons. I don't have any great ideas on how RubyGems profiles and GitHub profiles may be linked: storing the GitHub user id, perhaps?

To be clear, I still think the RubyGems IdP option needs to happen, perhaps as the default signing provider. If sigstore starts supporting profile-based signing, then work could begin on the various other parts of Phase 1 (i.e. opt-in) gem signing sooner. The RubyGems IdP work could start later, or progress in parallel.

[kerrizor]

A GitHub profile-based signature makes sense to me. It has the same pros as an RubyGems IdP, minus the vendorizing. It also sidesteps most of the cons. I don't have any great ideas on how RubyGems profiles and GitHub profiles may be linked: storing the GitHub user id, perhaps?

The con here, of course, being tied to a single commercial provider.

[rochlefebvre]

A GitHub profile-based signature makes sense to me. It has the same pros as an RubyGems IdP, minus the vendorizing. It also sidesteps most of the cons. I don't have any great ideas on how RubyGems profiles and GitHub profiles may be linked: storing the GitHub user id, perhaps?

The con here, of course, being tied to a single commercial provider.

GitHub profile signatures cannot be the only option, no. Having a GitHub account should not be a prerequisite for being a gem maintainer. That's why RubyGems profile signatures are still on the table. The latter take more effort to implement, and are trivial to forge in the event of a rubygems.org account takeover.

I'm suggesting that we look at supporting GitHub profiles first. That gets the ball rolling, but isn't sufficient for enabling opt-out gem signing (what the RFC refers to as Phase 2). We'd need a RubyGems IdP for that.

sigstore is the gatekeeper for supported identity providers. Really, any type of public profile that is meaningful to the gem ecosystem could work: GitLab, Twitter, etc.

[evanphx]

Hi everyone!

Lovely to see you all in here thinking about the future of rubygems security!

I just wanted to chime in and say that the rubygems IdP solution requires setting up federation with sigstore, which strangely enough is something that I did a few months back for a side project (hi @dlorenc :D ).

So I've already got a pretty good idea of how to code it up and it's not too difficult. The sigstore team has been great at helping get things up and going.

On the topic of encoding of ownership, I'd recommend that rubygems implement it via the spiffe type in fulcio, encoding the gem name as a spiffe url. This would provide gems the easily ability to see that the operation took place as part of the exchange with fulcio and rubygems.org.

[evanphx]

Oh, I didn't introduce myself. I'm Evan Phoenix, long time Ruby Central board member and long time rubygems.org admin. Currently the hosting bills for rubygems.org fall to me and thus a modicum of responsibility.

[trishankatdatadog]

Sorry, maybe I missed something, but has this RFC discussed how to handle developers who don't wish to sign their gems for whatever reason? I understand that there are phases in the RFC, but I can't imagine a world where everyone signs their own gems unless it's 100% mandated/required, and that can be difficult to achieve. Have you considered the option where RubyGems signs for gems by default unless developers provide their own sigs? (Similar to the PEP 480 model.) Again, apologies if I missed something somewhere.

[simi]

Sorry, maybe I missed something, but has this RFC discussed how to handle developers who don't wish to sign their gems for whatever reason? I understand that there are phases in the RFC, but I can't imagine a world where everyone signs their own gems unless it's 100% mandated/required, and that can be difficult to achieve. Have you considered the option where RubyGems signs for gems by default unless developers provide their own sigs? (Similar to the PEP 480 model.) Again, apologies if I missed something somewhere.

Per my understanding of https://github.com/Shopify/rfcs/blob/new-signing-mechanism/text/0000-introduce-a-new-signing-mechanism.md#guide-level-explanation, there is opt-out phase. You should be able to push the gem without signing using kind of CLI argument, but it will be enabled by default.

[jchestershopify]

Per my understanding ... there is opt-out phase

Correct, it would be an opt-out scheme. There would be an escape hatch but it would be as narrow and uncomfortable as we can make it. But eventually owners of most-downloaded gems should be required to sign, just as we're proposing that owners of most-downloaded gems should be required to enable MFA.

[jchestershopify]

Have you considered the option where RubyGems signs for gems by default unless developers provide their own sigs?

We foreshadowed that other kinds of log entries should be introduced in future, including a push / publish entry.

In my view this is functionally equivalent to the repository signing (just as gem owner signing can be seen as functionally equivalent to a build attestation). It's an example of where I would like different ecosystems to agree on a schema in the long run, but otherwise out of scope for this RFC.

[jchestershopify]

On the topic of encoding of ownership, I'd recommend that rubygems implement it via the spiffe type in fulcio, encoding the gem name as a spiffe url. This would provide gems the easily ability to see that the operation took place as part of the exchange with fulcio and rubygems.org.

Could you walk through an example? I've never felt like I grokked SPIFFE/SPIRE.

[trishankatdatadog]

Per my understanding of https://github.com/Shopify/rfcs/blob/new-signing-mechanism/text/0000-introduce-a-new-signing-mechanism.md#guide-level-explanation, there is opt-out phase. You should be able to push the gem without signing using kind of CLI argument, but it will be enabled by default.

Correct, it would be an opt-out scheme. There would be an escape hatch but it would be as narrow and uncomfortable as we can make it. But eventually owners of most-downloaded gems should be required to sign, just as we're proposing that owners of most-downloaded gems should be required to enable MFA.

Hmm, interesting. What's not clear to me right now is how client-side verification would work, especially with recursive dependencies. If I'm a package manager, do I turn on verification for all or no packages? How do I know which ones are supposed to be signed?

We foreshadowed that other kinds of log entries should be introduced in future, including a push / publish entry.

In my view this is functionally equivalent to the repository signing (just as gem owner signing can be seen as functionally equivalent to a build attestation). It's an example of where I would like different ecosystems to agree on a schema in the long run, but otherwise out of scope for this RFC.

Interesting idea about other kinds of log entries. However, I'm not sure that they are functionally equivalent to repository signing. As I mentioned, it's not clear for package managers which gems should be signed by developers vs repositories.

My advisor Justin Cappos and I looked at this issue a few years ago. We were very interested in finding strategies that would help effectively build the number of packages signed over time. In particular, you may be interested in Section 7 of our paper, where we used a month of downloads from PyPI to evaluate questions such as:

1. Assuming that an attacker controls PyPI for the whole month, what happens if all packages are signed by PyPI?
2. Now, what happens if the top 1% of projects (as measured by downloads) were signed by developers?
3. Now, what happens if we had also signed rarely updated projects (so it's not such a pain for developers to sign all the time)?
4. Now, what happens if we had also required projects to sign in order to upload a package?

And so on and so forth. Without going into the specific strategies, the figure above shows how we can reduce the body count, so to speak, as we adopt different strategies over time.

Sorry, I hope this wasn't a digression, but I wanted to get my thoughts out while they were fresh in my head. I hope this was helpful. Please let me know if you have questions. My main point is that I wanted to know: (1) how we convince projects to sign over time, and (2) how package managers can reliably know which projects should be signed by repositories vs projects/developers themselves.

[jchestershopify]

As part of the Securing Software Repos working group's efforts, @znewman01 published a short paper on different options for software repos in adopting sigstore. The paper canvasses a lot of what we've discussed in this thread so far, including whether to have RubyGems act as the IdP or continue to delegate it to others (including a possible non-vendor "neutral" IdP).

[sandstrom]

I think the overall idea of sigstore and not having long-lived keys is a good one. One thought on the issue of a single point of failure mentioned in the pros/cons summary above:

A new single point of failure - rubygems.org account takeover enables fraudulent signatures

If we keep the email or a RubyGems account as the primary identity/proof [signing subject], but also add support for secondary proofs.

These secondary proofs could be long-lived private keys stored in the users device or e.g. a Yubikey (old school private/public keys), or secondary attestations from another individual (via the proposed IdP flow, but this user wouldn't be the primary proof).

Basically, these secondary proofs would be allowed to change, but them doing so would be a useful signal. For example, if an author loses their private key for a secondary proof (new computer, etc) the gem client could show a warning. Popular gems could always have a another individual providing secondary proof, and if that person changes a warning would be shown.

It would be similar to the safety numbers used by Signal, or the device identifier that most SSO systems will store on a device (to keep track of device changes, and often ask for additional proof).

So, if someone have their account compromised at RubyGems, the attacker would have difficulty compromising the secondary proof (Yubikey or another persons account) at the same time. So a published gem update would draw more scrutiny if the secondary proof was missing or changed.

A legit owner replacing their secondary proof (computer or Yubikey) could use other channels, e.g. twitter or people they know, to communicate ahead of time that they are going to replace their secondary proof. All the while, their email and RubyGems account (primary proof) wouldn't change.

I know this is veering into protocol territory, so may be out of scope for this PR. But just wanted to mention it, since I like the overall idea of not using a third-party IdP. However, I also think the risk of account takeovers is real and could use more thought. This is one idea on how to reduce that risk.