Security Issue: IDP token to use only the users primary email #18046

Closed ChevronTango closed 5 months ago

Bug description

when running gp idp token then the JWT token that's generated has an email pulled, almost as random, from the users git auth provided. If a user adds an additional provider to their account, then they could potentially generate a token with any email they like. eg:

{
  "iss": "https://api.gitpod.io/idp",
  "aud": [
    "hello"
  ],
  "azp": "hello",
  "c_hash": "",
  "exp": 1687720947,
  "iat": 1687717347,
  "auth_time": 1687717347,
  "sub": "",
  "name": "ChevronTango",
  "email": "admin@example.com"
}

Steps to reproduce

Add a git auth provider to Gitpod and use it to log in. Then generate an idp token in any workspace in gitpod.

Workspace affected

No response

Expected behavior

in the above I added my own private GitLab instance to my account in Gitpod and then generated a token in a completely unrelated workspace. What I expected to see was a token bearing the email for my GitLab.com account, which I used to login to Gitpod. Instead I'm seeing the email from private GitLab instance, which has not been verified by Gitpod or a trusted upstream provider.

Example repository

No response

Anything else?

The email claim needs to have a restriction which means it is only ever pulling the users primary email (the one they logged in to Gitpod with). We should never trust any 3rd party git providers to supply an email which we pass as verified to an upstream service unless we have indeed verified it ourselves.

I know that the idp token system is still in BETA, but without this fault being addressed it would be impossible for any upstream system to trust Gitpod when a user can claim to be any email they like.

presently https://github.com/sigstore/fulcio/pull/1177 is blocked waiting on this, which would allow for some keyless commit signing as outlined in https://github.com/gitpod-io/gitpod/issues/666

this follows on from the work introduced in https://github.com/gitpod-io/gitpod/issues/17530 and https://github.com/gitpod-io/gitpod/pull/17678

The logic in question which is at fault is https://github.com/gitpod-io/gitpod/blob/main/components/public-api-server/pkg/apiv1/identityprovider.go#L74-L80 which is where the loop iterates over all the identity providers the user has attached to their account, and returns only the first email it finds, with no further validation or checks.

I suspect the assumption was that all the emails would be the same for all the providers but that's not the case. Instead this should be first filtered to only Gitpod managed providers, and if possible further filtered to only the email of the provider of the current workspace. Care should be taken to make sure we support Dedicated users who may have different base auth providers to gitpod.io. We probably don't want to return an email for any workspace launched against a different git provider, though it could be argued that we always should return the account email regardless of the workspace which is open. Its an interesting debate and it could be argued either way.

@loujaybee Would it be possible to get an update on this, and also the general state of the JWT work please? I note that my PR on Fulcio are blocked on this and I'm very excited to get this in a state where it starts benefitting users.

@loujaybee, I've just seen that the OIDC feature has been announced via a new changelog announcement, but to my knowledge this security issue still exists. If users start using Gitpod as an identity provider there is a risk that others could impersonate them by configuring additional OAuth providers for their account. Can you confirm that this security issue is a priority considering the OIDC feature is now in production for general use?

Hey @ChevronTango, thanks for the patience, I have been meaning to get back to you.

Emails in Gitpod Cloud are slightly differently than in Gitpod Dedicated right now. Over the fullness of time we are working to bring these behaviours closer together. Gitpod remains one product with two offerings, but in Gitpod Dedicated there is only one organisation per Gitpod installation and with SSO the organisation controls the users (and therefore controls the emails that are set via SSO) which simplifies some things regarding this feature.

In Gitpod Cloud because the user (not the organisation) controls their own SCM integrations, and the emails originate from those SCM integrations for the JWT there is a chain of trust. However, because Gitpod cannot guarentee that an email provided by a third party is truly verified Gitpod Cloud does not include the email_verified claim.

This is mentioned in the docs here: https://www.gitpod.io/docs/configure/workspaces/oidc.

There are a couple (or multiple fixes) to get verified email claims working for Gitpod Cloud...

Verify all emails regardless of their source so we can set email_verified true.
Transfer ownership of the email and user to the organisation for Gitpod Cloud.

I'd love to unblock you @ChevronTango but I can't say when this will be resolved right now. We are making a lot of improvements to give more control to organisations and administrators including better control and visibility into users within an organisation which hopefully will address this feature/issue.

Hopefully that answers your question ! 🙏

Thanks for getting back to me @loujaybee, and thanks for clearing up some of the logic in this area. My concern right now is less for the blocked PRs I have and more for the users on Cloud who might be inadvertently trusting this feature in spite of the above. If they have setup AWS or Vault integration with a reliance on Gitpod Cloud's JWTs, they may well be vulnerable to an impersonation attack via this approach. The email_verified claim is important, but not all downstream services that users have configured will use it. That being said, I whole heartedly support the effort to restore it, and its temporary removal for now.

As for you solutions, I think I outlined my preferences above. I think the email field should only be returned if the workspace is from an accepted provider (GitLab, GitHub, Bitbucket etc on Cloud and whatever the users want for Dedicated). Any user specified provider on Cloud cannot be trusted and so shouldn't return any email claims ever. I also think you need to harden the logic so the provider matches the one used to launch the workspace, rather than being chosen at random from the list. If both of those were in place we'd be fine, and you wouldn't have to do any additional verification yourselves.

I don't think you need to verify an account wide email with this approach, though that would be a clearer alternative and if you choose to go down that road, it would be a welcome solution. I also don't have much of an opinion on organisations owning the email side of things either.

To bring it back to my main point, right now, on cloud, the logic is flawed and users should not trust the JWT tokens email field. I don't know how many users are using this right now, or what setups they might be using, but this remains a vulnerability that can be exploited and I really think it should either be patched or the users informed so that they don't open themselves up to attack. I can well imagine a user or team integrating with Vault and they'd be very keen to know if there was a way for an attacker to impersonate them and access their secrets.

Hello @loujaybee. I was wondering if there had been any movement on this issue? Really keen to get sigstore integrated with gitpod natively. I appreciate the complexity of the issue but is there any plans to add email verification formally into gitpod?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

gitpod-io / gitpod