Open kvedes opened 4 months ago
Found this issue which proposes a solution I would solve my issue: #12049
Yes that solves a number of issues. I'd like to implement that too, but need a solid chunk of time to understand it in-depth, implement, and test. Argo also might still need to issue its own tokens as a fallback in some cases (I'm not sure yet).
TLDR; Argo SSO doesn't encode anything from the Azure token into its' own token, thus use of public key is enough to authenticate.
I need to take a closer look at the rest. If accurate, only needing the public key would certainly be problematic in a few ways. Although currently, as you stated, since Argo generates both in a k8s Secret there would be no new exploit route (would still be a vuln, but 0 impact vuln).
Argo does validate the OAuth2 token before exchanging for its own token. Bypassing that and crafting your own tokens with the private key would be fair game for a token exchange (although now I'm wondering if capability/authZ-wise there's potentially a MITM possibility in and of itself there? π€).
Btw, Argo does have a private security disclosure process. A bit late at this point though π
Sorry this is my first time reporting a security issue, so I hadn't thought about people misusing it. Feel free to take down the issue and keep it internally.
Sorry this is my first time reporting a security issue, so I hadn't thought about people misusing it
For future reference, please use projects' SECURITY.md
or other publicly posted security contacts. On GitHub, the UI links to it in a few places and it's also present in the /security
route of any repo, for instance https://github.com/argoproj/argo-workflows/security.
For websites there's also security.txt
, typically under /.well-known/security.txt
, for instance https://www.google.com/.well-known/security.txt.
Feel free to take down the issue and keep it internally.
A little bit easier said than done, I believe this requires a GH support request π. As this is already public, it's likely already been scraped by crawlers too, so taking it down is not quite so useful.
With regard to the actual contents of the issue:
The code only uses the public part of the sso key.
So your code uses the private key, not the public key. Could you elaborate why you think only the public key is necessary?
Bypassing that and crafting your own tokens with the private key would be fair game for a token exchange (although now I'm wondering if capability/authZ-wise there's potentially a MITM possibility in and of itself there? π€).
This token exchange does feel a little weak, but the only MITM is if someone has access to your Argo Server's private key, they can then potentially access everything that all users can access. That would normally require the same permissions as to modify the Argo Server's Deployment though, if not even more elevated (as it would require permissions to Secrets in Argo's namespace), so there's technically no privilege escalation route there. If you can modify the Server, you can turn off auth entirely
So the key stored in the Kubernetes secret sso contains both a private and a public key. My point is, that if you extract the public key, that is sufficient to create a token. This means that if at some point the key is split into a private and a public key, and the public part is exposed there will be a security hole.
My code example might be little misleading, due to my lack of Go skills. I use the "private key" in the example, which is the full key from Kubernetes which contains both the public and private key as mentioned above. The code then extracts the public part of the key:
encrypter, err := jose.NewEncrypter(jose.A256GCM, jose.Recipient{Algorithm: jose.RSA_OAEP_256, Key: privateKey.Public()}, &jose.EncrypterOptions{Compression: jose.DEFLATE})
Note the part that extracts the public key: Key: privateKey.Public()
I did manage to manually extract the public part of the key, but as mentioned my Go skills are limited so I'm not sure how to pass that in directly. The above code is based on the code from the sso module in Argo Workflows.
Note the part that extracts the public key:
Key: privateKey.Public()
Oh I missed that detail (it requires scrolling to the right also). I've edited your initial sample to clarify that in the comments.
I'll have to look into this more why it's accepting that π
I'll have to look into this more why it's accepting that π
Oh... the code straight up uses the public key to encrypt: https://github.com/argoproj/argo-workflows/blob/5b7d1472f393d60d884d3809b57bde7d4d7bd538/server/auth/sso/sso.go#L182
That seems to date back to #4095 and seems to have been on purpose, as the tokens are opaque. I'm not sure if that's necessary, but that would explain usage of the public key; only the private key can decrypt it.
Looking further, Argo having its own JWE apparently stems from #4035 / #4027 / #3873 et al. Instead of implementing refresh tokens in that PR, it was decided to use a new JWE effectively as a token exchange. Although it's not entirely clear why that was decided.
I might be missing something, but that would suggest that #12049 was the original approach and then was changed?
Although currently, as you stated, since Argo generates both in a k8s Secret there would be no new exploit route
Oh it doesn't generate both, it only generates the private key in the Secret. The public key is then derived from the private key if I'm understanding the code correctly.
That's a bit better, technically the public key is not stored anywhere.
This is effectively doing symmetric encryption but using asymmetric keys, which is confusing and odd. It would probably be more correct to just use symmetric encryption
It seems that crypto/rsa generates a private key that also contains a public key: rsa source code. See line 113-127.
As to your considerations about encryption options I'm not sure what the best option is. Just seems weird to me, that Argo WF issues its' own JWE tokens, instead of validating the SSO tokens from the provider.
Just seems weird to me, that Argo WF issues its' own JWE tokens, instead of validating the SSO tokens from the provider.
Just to be 100% explicit, as I wrote before, Argo does validate the provider tokens and then exchanges those for its own longer-lived encrypted/opaque tokens. (confusingly asymmetric as symmetric)
Otherwise I agree that directly using the provider tokens would seem to make more sense (and have less indirection and therefore less room for error), and some SIG Security folks agreed (albeit only based on a short summary) during a meeting today as well.
I'm not sure if you saw my comment from earlier today -- apparently Argo did that in the past and there was support & progress to improve that, but then it got replaced with the current JWE approach without much documented explanation.
I'd like to make sure we have full context on that decision before changing that. Off-hand, I can imagine the JWE could be useful for some providers that don't support refresh tokens or have other limitations, but I would imagine that the vast majority are capable, and that Dex as an intermediary could mitigate various limitations too (if needed, similar to its current optional usage in Workflows). May check with some Intuit folks if they can find records of it etc
Followed up on this in the May 28th SIG Security meeting. Did get some Intuit folks to check around for records with no success unfortunately.
We also discussed CD's approach (caveat: it has a lot of different options), which in one variant it just signs tokens and doesn't encrypt them.
This is potentially applicable to Workflows if the contents of the token do not need to be secret (which may be the case -- it might have been a remnant of the access token approach PR #4035). Then we'd just validate the signature and wouldn't need to encrypt/decrypt, which would make this issue no longer applicable. That would also be a correct/more appropriate use of asymmetric cryptography and so would be (substantially) less confusing than the current "asymmetric as symmetric" approach (the confusion of which resulted in this issue).
Pre-requisites
:latest
image tag (i.e.quay.io/argoproj/workflow-controller:latest
) and can confirm the issue still exists on:latest
. If not, I have explained why, in detail, in my description below.What happened/what did you expect to happen?
I have SSO configured with Azure Entra ID. I have experimented a bit with the authentication, and it seems to me, that something is wrong in the SSO flow. Argo seems to discard all information in the token received from Azure before it issues its' own token. I have created some simple Go code to illustrate it. If I extract the
sso
secret created by Arog Workflows and run the code below, I can create a token which will successfully authenticate. The code only uses the public part of the sso key. Note that I put in an object ID from an Entra ID group which is allowed access through the Azure Enterprise App which Argo uses for authentication (putting 000-.. as an example). Such an ID is easily obtained by people inside an organization, as Entra object ID's are not secret .This means that obtaining the public key is enough to authenticate with Argo when SSO is enabled. I realize that the
sso
secret is not publicly accessible, however it seems to me that this should not be possible regardless. My knowledge of authentication is limited, but I would expect to see some of the properties from the Azure token contained in the Argo token. In that way Argo can verify that the client has actually authenticated with Azure beforehand.Found this issue which proposes a solution I would solve my issue: https://github.com/argoproj/argo-workflows/issues/12049
TLDR; Argo SSO doesn't encode anything from the Azure token into its' own token, thus use of public key is enough to authenticate.
Version
v3.4.11
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
Logs from the workflow controller
Logs from in your workflow's wait container