Closed mvalenzisi closed 1 year ago
@weinong could you please have a look at this request?
I have a dream of making all OIDC related providers be based on the OIDCProvider
struct in their implementation. Similar to this refactor we are making for Keycloak here: https://github.com/oauth2-proxy/oauth2-proxy/pull/1210
It drastically simplifies the management of the Provider ecosystem and allows us to add core improvements to OIDC that can apply to all OIDC based providers.
Can someone clarify the state of the Azure versions (I'm not an Azure user). I think there was design decisions to make the Azure provider we have today a swiss army knife and apply to OIDC & non-OIDC versions. But that made it unable to embed the OIDC provider and inherit its implementations.
If that is a v1 vs v2 thing -- I think we should split the Azure provider and support V2 in a new azure-v2
provider that is completely OIDC based and look to sunset azure
.
CC: @codablock & @weinong since you both have worked in this Provider most recently.
And that's an awful large number of 👍 so quickly... Did you crowdsource coworkers? 🤣
@NickMeves I understand your desire to have a single OIDC provider for everything but there are few contingent problems:
ADAL JWT issue:
-api-version=1.6
, required by the graph server.I don't trust Microsoft regarding their compliance with protocols. 😉
In conclusion, there is currently no way to use AAD as an OIDC provider in any way in clusters that are not managed directly by Azure.
Ref: https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-migration https://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims
oauth2-proxy supports Azure AD v2.0 via the OIDC provider. There are just a few things that need to be taken care of.
https://login.microsoftonline.com/<tenant-id>/v2.0
resource
in v1. If for example, you want to get tokens for your own exposed API, use <client-id>/<api-name>
. Also don't forget to add the other/default scopes, e.g. openid
and profile
.If you leave out 2., you'll end up with tokens for the Graph API. These are very hard to validate and actually not meant to be validated at all by you (Graph API validates them).
When you get your own API tokens (by following 2.), you'll then have another disadvantage: oauth2-proxy can't perform profile requests anymore, as these only work with Graph API tokens. This means, you'll only get the information inside your tokens which are part of the initial token claims.
Take a look at either of these two PRs trying this new architecture:
ADFS: https://github.com/oauth2-proxy/oauth2-proxy/pull/1021 Keycloak: https://github.com/oauth2-proxy/oauth2-proxy/pull/1210
(ADFS in particular might be very related in how it handles nuances of scope
in Azure/MS world).
Any areas where we need to deviate due to custom behavior of the provider, we can override those methods. But otherwise reduce code duplication by using default OIDC behavior where applicable.
Would that strategy work for Azure?
@NickMeves I understand your desire to have a single OIDC provider for everything but there are few contingent problems:
@pierluigilenoci My goal is to have the OIDC provider as a base implementation for OIDC based providers (rather than bare bones OAuth2 at the moment that is the base of all providers). Azure would still have its own provider with code handling all the Azure special cases that deviate from standard OIDC.
In Kubeapps we have some users looking forward to using the managed AAD for login in their AKS clusters. It was tricky to come up with a configuration that worked (eg. https://github.com/oauth2-proxy/oauth2-proxy/issues/1144) but we finally figured that out!!
In case it's useful, here is a step-by-step guide on how we set it up: https://github.com/kubeapps/kubeapps/blob/master/docs/user/using-an-OIDC-provider.md#aks-managed-azure-active-directory
Key aspects:
oidc
provider, not the azure
one (it even crashed in some cases I wasn't able to fully identify)accessTokenAcceptedVersion
to 2
--oidc-issuer-url
) is not sts.windows.... but this one: https://login.microsoftonline.com/<MY-TENANT-ID>/v2.0
--scope
) instead of the --resource
and it should be [for login into AKS]:
--scope=openid email 6dae42f8-4368-4678-94ff-3960e28e3630/user.read
edit: I forgot to add the email in the scope. It is also required (unless you wanna include it as part of the optional claims)
Hope it helps someone :P The good news is that we can confirm the oidc
provider is perfectly working for us using v2
tokens.
- The issuer (
--oidc-issuer-url
) is not sts.windows.... but this one:https://login.microsoftonline.com/<MY-TENANT-ID>/v2.0
- And the crux... we are using the scope (
--scope
) instead of the--resource
and it should be [for login into AKS]:--scope=openid 6dae42f8-4368-4678-94ff-3960e28e3630/user.read
Thanks for raising these 2 details!
If I recall, in the implementation of the ADFS provider (very similar to Azure, OIDC-based), we had explored the potential to dynamically construct scope & issuer URLs for users based on a tenant-id
.
Seems like that is potentially still a viable strategy when making an azure-v2
that is OIDC based.
Historical Thread in ADFS PR: https://github.com/oauth2-proxy/oauth2-proxy/pull/1021#discussion_r571505330
@antgamdia I tried your way to configure OAuth2-Proxy:
--http-address=0.0.0.0:4180
--cookie-domain=<OUR_DOMAIN>
--cookie-expire=1h
--cookie-refresh=59m
--email-domain=*
--exclude-logging-path=/ping
--oidc-email-claim=sub
--oidc-issuer-url=https://login.microsoftonline.com/<OUR_TENANT>/v2.0
--pass-access-token=true
--pass-user-headers=true
--provider=oidc
--provider-display-name=<COMPANY_NAME>
--redirect-url=https://<AKS_URL>/oauth2/callback
--scope=openid email 6dae42f8-4368-4678-94ff-3960e28e3630/user.read
--set-authorization-header=true
--set-xauthrequest=true
--whitelist-domain=.<OUR_DOMAIN>
--config=/etc/oauth2_proxy/oauth2_proxy.cfg
Our AAD App manifest:
{
"id": [REDACTED],
"acceptMappedClaims": null,
"accessTokenAcceptedVersion": 2,
[REDACTED]
}
I'm able to log in and I get this JWT:
{
"aud": "6dae42f8-4368-4678-94ff-3960e28e3630",
"iss": "https://sts.windows.net/<OUR_TENANT>/",
[REDACTED]
},
"_claim_sources": {
"src1": {
"endpoint": "https://graph.windows.net/<OUR_TENANT>/users/[REDACTED]/getMemberObjects"
}
},
[REDACTED]
"scp": "user.read",
[REDACTED]
"ver": "1.0"
[REDACTED]
}
It seems that the token is still v1.0, also the URLs are without /v2.0
and the issuer is sts.windows.net
.
What do we do wrong?
@codablock could you please take a look?
@NickMeves I agree with your intentions but I don't think Microsoft is collaborative 😝 @weinong could you please help us?
@codablock @NickMeves @weinong @antgamdia I'm sorry to bother you but I need some ideas to solve this problem. 🙏🏻 ❤️
I don't have much experience in Azure (I just had to jump in to help our Kubeapps users), so I'm sure other people would be much more useful. My use case: I needed to enable users to authenticate to their AKS clusters using a UI that uses oauth2proxy behind nginx.
That said, what I did to try to debug my problems was using the token in my kubeconfig as my target example. That way, I would compare the token I got with the token I should be getting. Note I'm using that approach because I need to get access to an AKS cluster.
users:
- name: clusterUser_cc_az_webhook_resource_group_my-resource-group
user:
auth-provider:
config:
access-token: >-
eyJ0eXA...
Then, I pasted it in https://jwt.ms/ and compare it with the one generated by oauth2proxy.
For instance, in my case, the issuer is "iss": "https://sts.windows.net/<MY_TENTANT>/"
but I don't have any _claim_sources
(so my token is v2 I guess).
Also, I noticed you're using --oidc-email-claim=sub
, which I'm not (but I don't really know if it required for you or not)
I'd try creating a new app registration in azure and checking the guide we wrote for our Kubeapps users. But, as I said, I'm not an Azure expert at all :P
Hope you get your issue solved soon!
@pierluigilenoci I can't say for sure, but the scope 6dae42f8-4368-4678-94ff-3960e28e3630/user.read
looks suspicious. It looks like you're requesting some Graph or AD scope. Did you try to use your own scope, e.g. <my-apps-client-id>/<my-custom-scope>
? API scopes can be added in "Expose an API" in the app registration config.
but the scope
6dae42f8-4368-4678-94ff-3960e28e3630/user.read
looks suspicious.
No, this odd scope is precisely the required one for request the protected resource "Azure Kubernetes Clusters". I know it's seems arbitrary, but it is always the same.
You can check it by executing:
az ad sp list --display-name "Azure Kubernetes Service AAD Server"
This way, you will get the app id corresponding to the "Azure Kubernetes Service AAD Server". The next thing is user.read
, which is the actual scope.
So, as far as I can tell, 6dae42f8-4368-4678-94ff-3960e28e3630/user.read
is the right scope to ask if you need access to AKS.
@NickMeves after a long and debilitating discussion with AAD's Azure Support I came to the conclusion that as of today there is no _claim_sources
endpoint for groups that is OIDC compliant.
At this point there are two possibilities:
@antgamdia let me explain in more detail. Even configuring OAuth2-Proxy to use the 2.0 version of the OIDC issuer endpoint, the returned JWT token is of type 1.0 because the version of the token depends on the scope
used. To date, there is no compatible scope
to having groups via a Microsoft Graph endpoint (that is, a compatible OIDC endpoint). The token version is however explicitly written in JSON.
For completeness of the information:
I would like to have @weinong opinion on the matter given his internal position.
I have to admit the conversation about ADAL tokens is going over my head, but in summary, it sounds like we can use the existing OIDC provider only if the returned tokens contain all the data required to evaluate which groups a user belongs to, etc. If we want to make Microsoft Graph API calls to get the data later we apparently cannot use the token provided by the OIDC provider? I'm a bit confused about the necessity of doing this, but I can see why it might be required.
All I can follow up with is that I managed to get the OAuth2-proxy OIDC provider to work with and return either v1 or v2 tokens out of the box, but that I never tried using any of the Graph API or what's being spoken about for Kubernetes compatibility. If there's a way to make this work with v1 tokens but not v2 tokens, it's possible to get v1 tokens using the OIDC provider. But again, I included groups in my tokens and didn't need to make additional Graph API calls.
@LouisStAmour the _claim_sources
field is valued with the groups in the JWT token only when the groups are more than 200, it is a problem only in large and/or complex organizations: less than 200 groups the token works, more than 200 groups the token is broken.
To ensure that the token size doesn't exceed HTTP header size limits, Azure AD limits the number of object IDs that it includes in the groups claim. If a user is member of more groups than the overage limit (150 for SAML tokens, 200 for JWT tokens, and only 6 if issued via the implicit flow), then Azure AD does not emit the groups claim in the token. Instead, it includes an overage claim in the token that indicates to the application to query the Microsoft Graph API to retrieve the user's group membership.
Ref: https://docs.microsoft.com/en-us/azure/active-directory/develop/access-tokens#payload-claim
On the Azure side, there is no way to filter the groups in advance. I had a long conversation with Azure support for this as well.
And obviously, companies cannot be asked to make their governance or the management of internal permissions less complex. 😜
The current JWT token that is returned with version 1.0 (ADAL JWT) is NOT compliant with the OIDC protocol and therefore any external software is unable to use it. See my previous comment for details.
To date, even using the new MSAL 2.0 endpoint does not solve the problem because the version of the token returned is always the 1.0 version because the token version does not depend on the version of the endpoint being called. See my previous comment for details.
I hope I have clarified the situation for you.
Just to give some more information Hashicorp solved it this way: https://github.com/hashicorp/vault-plugin-auth-jwt/pull/120
The only users who are not affected by this problem are users who use AKS to run Kubernetes because the clusters managed by Azure are able to manage the JWT token that Azure produces. 🤣
If you try to use AAD to authenticate yourself on a vanilla K8S (or for example AWS EKS) you are authenticated but not authorized because RBAC is not able to know which group you belong to (in case the roles are assigned via groups of course).
Another discussion about this https://github.com/kubernetes/kubernetes/issues/62920
@NickMeves could you take a look at the discussion?
Sorry for the very very late reply. Been too busy on something else. @pierluigilenoci , if I understand the problem correctly, you are trying to get a JWT token for AKS AAD Server App, and regardless how you configure, it always return v1 token? This is expected as the token format is determined by the server app, which is AKS AAD Server App.
ADAL JWT issue: The graph API URL embedded in the OIDC token doesn't have a parameter -api-version=1.6, required by the graph server. The JWT token doesn't have access_token embedded, but is required by the graph server.
@pierluigilenoci I don't quite follow this. When you get the access token from v2 endpoint for https://graph.microsoft.com/
using oidc provider, you should get a valid token for Graph api.
@pierluigilenoci I'm just lurking -- I'm not an Azure user, so I don't have the subject matter expertise to make a PR for this.
I'm following along to be ready once someone takes the plunge and makes a PR.
My preference:
Base the Azure provider off the OIDCProvider. If v2 is significant'y different than v1, don't try to support both in the same provider -- make 2 providers (e.g. azure
and azure-v2
)
@weinong actually what I tried to do is to use MSAL because Microsoft promised it was OIDC compliant (ADAL is not by their own admission). But even using MSAL the JWT token is version 1.0.
However, whatever endpoint and configuration I use I always get a "malformed" JWT token because the _claim_sources
is not usable. It is not usable because the URL is incomplete and the access token is missing. Ultimately the _claim_sources
(and therefore the JWT token) that Azure provides is not OIDC compliant as it should be. The JWT must be valid in fact, not just formally. [1]
I repeat it here for completeness.
ADAL JWT issue:
-api-version=1.6
, required by the Graph server.So Kubernetes (other than AKS) and any third-party OIDC client will not be able to pull out groups if the user has more than 200. Some tools (few) succeed because they use workarounds, not the strict protocol.
I really don't know how to write it any clearer than that.
@NickMeves if you are not an Azure user, feel lucky. 😜
[1] https://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims
I see. Your comment is mainly towards how AAD handles overage claim, that I cannot help you with as it seems to be by design from AAD's perspective. (don't quote me on that as I'm not from AAD team :P)
Regardless, what claim do you need in order to use oidc
provider so that you don't query the Graph api?
Hi @weinong, thank you for your time.
From my point of view, the fact that Microsoft does not respect the OIDC protocols "by design" sounds like a kind of joke. 😜
The only claim I need is the one that returns the user groups. Nothing more. 😞
But above 200 groups JWT does not return the groups directly but provides a claim that does not respect the protocol.
I think it's related to #1073 too
Hi folks, just came across this issue and I can tell that @pierluigilenoci solution worked for me. But this solutions or specifically the endpoint is not the same as described in the Azure documentation here.
For oauth2-proxy we need to use: https://login.microsoftonline.com/
Using the Azure documentation will lead into a crashing oauth2-proxy pod with the log message:
[main.go:54] 404 Not Found:
For oauth2-proxy we need to use: login.microsoftonline.com
/v2.0 While Azure says: login.microsoftonline.com /oauth2/v2.0 Using the Azure documentation will lead into a crashing oauth2-proxy pod with the log message:
[main.go:54] 404 Not Found:
This story is becoming more and more like a joke... 🤦🏻♂️
It's just a minor inconsistency. ;) Not sure though, what happens on Azure site by calling the wrong endpoint. I guess there is a fallback so it still works but it should definitely be implemented.
We are really happy with the oauth2-proxy, just together with Azure it is a little more complicated and we'd like to have a support for the /common
endpoint but thats a feature request and doesn't belong into that issue.
Have a great day.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
Hi Mr GitHub-Actions could you please remove the stale? 👅
FYI - Someone in the community will need to pick this up.
Neither @JoelSpeed nor I are Azure users and our free time to support the project lately is drastically reduced.
I have been playing around locally and using the OIDC not azure provider seems to work for my needs with azure v2.0 endpoint: A backend API and secure it passing on jwt access token which contains the users roles/groups.
I think the key part is having a 2nd app registration in azure so that you can get access tokens for your backend instead of for graph. So I use the first app registration for authentication and that gives access to graph and the second app registration for my protected resources / access tokens for my backend.
The ouath2-proxy config key parts are:
provider="oidc" scope="openid email api://<app reg 2 / expose an api / Application ID URI>/<scope you created e.g. access_as_user>" oidc_issuer_url="https://login.microsoftonline.com/<tenant id from>/v2.0" client_id="<from app reg 1 secrets>" client_secret="<from app reg 1 secrets>" pass_access_token = true //access token that gets passed to the backend containing roles etc.. oidc_groups_claim="roles" //Could set this to groups if in azure we tick the box to emit roles as groups claim
you can also restrict access via oauth2 proxy using ad roles and groups (will need to emit them in azure) --allowed-group="staff"
Set accessTokenAcceptedVersion property to 2 in the azure app registration manifest.
These instructions (kind of) show you how to setup the two app registrations in azure and delegate access it is intended for a different purpose but the steps are similar. https://github.com/AzureAD/microsoft-authentication-library-for-js/tree/dev/samples/msal-node-samples/on-behalf-of#Register
The following sections in the link shows the Azure app registration config and delegation: Register a web API Register a web app
It would be good to get some others to test this configuration and raise a full config example.
Hi folks, just came across this issue and I can tell that @pierluigilenoci solution worked for me. But this solutions or specifically the endpoint is not the same as described in the Azure documentation here.
For oauth2-proxy we need to use: https://login.microsoftonline.com/
/v2.0 While Azure says: https://login.microsoftonline.com/ /oauth2/v2.0 Using the Azure documentation will lead into a crashing oauth2-proxy pod with the log message:
[main.go:54] 404 Not Found:
The Azure documentation is correct. oauth2-proxy uses the endpoint without the /oauth2 path segment as the token issuer and to retrieve the openid configuration https://login.microsoftonline.com/
We have also switched from the azure
provider to the OIDCProvider
for connecting to the Microsoft Identity Platform v2.0 endpoints. Our use case is the simplest, with only web applications being fronted by the oauth2-proxy.
The relevant configuration items are:
provider = "oidc"
oidc_issuer_url = "https://login.microsoftonline.com/<REDACTED-TENANT-ID>/v2.0
and the client ID corresponding to the AAD App registration set up in Azure. For web applications, I think the client secret is not even necessary because validation is performed on the redirect/callback URL where the token is posted back. The client secret would still be needed for the authorization code flow.
Similar to the use case of @LouisStAmour above https://github.com/oauth2-proxy/oauth2-proxy/issues/1231#issuecomment-872583808, all the claims we needed are available in the initial token claims and there wasn't a need for the OIDCProvider
to call additional APIs (e.g., graph.microsoft.com) to retrieve more user information. Our use case doesn't utilize more granular permissions on our application; the only checks we require are role/group membership and OIDCProvider
is sufficient for this.
For use cases where information from Microsoft's Graph API is needed, then the azure
provider (or the proposed azure-v2
provider) may have to be used instead of the base OIDCProvider
. Or maybe the profile-url
configuration can be used as a generic way to retrieve additional user information. I also see that there may be issues with using the resource
configuration with the azure
provider described in #1144.
The iss
claim in the token comes back correctly as https://login.microsoftonline.com/<REDACTED-TENANT-ID>/v2.0
. Additionaly, a roles
claim containing the app role memberships for the user are automatically included in the initial token claims.
We identified one nuance with the base OIDCProvider
. Another reason we switched from the azure
provider to this base OIDCProvider
is because of the open issue #888. If your oauth2-proxy configuration makes use of allowed_groups
as a means for verifying group/role membership, and you do not explicitly set the scope
configuration, then the base OIDCProvider
will use a default openid profile email
for the scope
configuration, but also automatically append an additional groups
in this scope
configuration. This was code added as part of #616. IMO, that snippet of code:
if len(o.Providers[0].AllowedGroups) > 0 {
o.Providers[0].Scope += " groups"
}
should not have been put in the base OIDCProvider
but just replicated into specific providers. groups
is not one of the standard scopes defined in the OIDC specs https://openid.net/specs/openid-connect-core-1_0.html#ScopeClaims.
If an oauth2-proxy implementation using the base OIDCProvider
wants to request the groups
claim, it probably will want to use the proper parameters in the authorization request as defined in https://openid.net/specs/openid-connect-core-1_0.html#ClaimsParameter
In our oauth2-proxy implementation, we had to explicitly set the scope
configuration to openid profile email
to prevent the base OIDCProvider
from appending the groups
scope in the request.
It seems that Microsoft's v1.0 endpoints merely ignore the unrecognized groups
scope in the authorization request. The v2.0 endpoints return an invalid_client
error with a description of AADSTS650053: The application asked for scope 'groups' that doesn't exist ...
when it encounters the unrecognized groups
scope request. In the OIDC specs, there wasn't a prescribed behavior when getting an unrecognized scope request. The OAuth2 specs https://datatracker.ietf.org/doc/html/rfc6749#section-3.3 also neither prescribes an error condition nor an ignore behavior when encountering an unrecognized scope request.
Am I correct in understanding that adding for example "given_name" and "family_name" claims to the token does not make this information available to oauth2-proxy? (And this information needs to be requested through graph API calls?) I've tried to add "optional claims" on the Azure AD App Registration under "Token Configuration" - and I have been successful in adding groups there, but any other claims I add seem to just get ignored and don't result in additional headers being sent to my app.
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
My hopes of seeing this work are getting fainter... 😭
The problem is getting bigger #1505 (also #1144 is involved)
@weinong could you please take a look again at this?
This issue has been inactive for 60 days. If the issue is still relevant please comment to re-activate the issue. If no action is taken within 7 days, the issue will be marked closed.
Still an issue
@weinong could you please take a look again at this?
CC: @braunsonm
I believe you have some more Azure background than @JoelSpeed or I to potentially help with this (and triage the criticality of the work).
@NickMeves This is important but as noted by many here, you can somewhat get around this using the oidc
provider instead of Azure. This is what I currently do. The only limitation I'm aware of is you do not get the full list of groups if you have hundreds of groups which is what: https://github.com/oauth2-proxy/oauth2-proxy/pull/1574 is addressing.
I think that linked PR is our best bet to implement this. I have not seen any mention from MS that they are deprecating v1 (AAD Endpoints) unlike what OP claims in the original post. Though they don't recommend new projects use it.
@adriananeci 's PR is the closest right now and would be nice to have since the MS Identity Platform is the only one that supports PKCE as well which we are going to support in the next release 😄 I can take another look at that PR but I had issue with not using the OOB flow to exchange the token in order to talk to MS Graph.
Trying to use this in a multitenant setup where the azure endpoint is https://login.microsoftonline.com/organizations/v2.0 and it won't start with a failure saying expected "https://login.microsoftonline.com/organizations/v2.0" got "https://login.microsoftonline.com/{tenantid}/v2.0"
I was able to successfully get it to work with the oidc provider against Azure for multitenant. A couple of things that were causing me trouble; the skip issuer verify for multitenant and the oidc email claim. I also set the token to v2 in the app manifest
- --provider=oidc
- --oidc-issuer-url=https://login.microsoftonline.com/organizations/v2.0
- --insecure-oidc-skip-issuer-verification=true #required for azure multi-tenant
- --scope=openid email profile
- --oidc-email-claim=preferred_username
@JoelSpeed I believe this was closed by mistake. Can it be reopened?
At the moment the Azure provider supports only Azure Active Directory (v1.0) endpoints. Version v1.0 is deprecated by Microsoft and not fully compliant with the OIDC protocol.
Expected Behavior
The azure provider should be able to retrieve a JWT token using the v2.0 endpoint.
Current Behavior
At the moment if you use a v2.0 endpoint you get the error:
Steps to Reproduce (for bugs)
This should be a working configuration:
Your Environment