Add support for Vault Unwrap Token

lordofthejars commented 4 years ago

Description

Add support for what is known as Cubbyhole Response Wrapping.

Implementation ideas

The idea is just to provide support for the unwrap operation of a wrapped token to authenticate. So basically you can support the next use case:

Create a token with default policy
Authenticate with Vault using this default token (less privileged token)
Unwrap the secret to obtain more privileged token (apps persona token) and authenticate with it.
Verify that you can read secret/dev using the appstoken

So basically I need to be able to do 2 and 3. The second point is already supported as token authentication is working, but I need to add the operation of unwrapping the token and then reauthenticate automatically with the unwrapped token.

Feel free to assign me this issue.

lordofthejars commented 4 years ago

Needs https://github.com/quarkusio/quarkus/pull/7891 to be merged before continuing with this one to avoid merge conflicts.

lordofthejars commented 4 years ago

@vsevel I think it is really interesting to use case to cover, probably not if you deploy to Kubernetes as we already have support for Kubernetes auth, but for other cases.

sberyozkin commented 4 years ago

@lordofthejars Hi, why would one prefer this method, from a practical point of view, to the current authentication non-kubernetes default method or compared to a pretty strong TLS Cert method ?

lordofthejars commented 4 years ago

Well with none Kubernetes you are setting a token in the code, so anyone could steal the token and has access to everything. With unwrap, you are providing a single-use short life token, then you use this wrapped token to unwrap another token which has the real access to secrets.

Then if someone steals this token since it is a single-use token when the real app (legitime) tries to use it then it will receive an error that the token has been already used. Since this should not happen, you can warn the security team that someone has stolen a token so halt any security operations.

sberyozkin commented 4 years ago

@lordofthejars Makes sense, IMHO can be of interest to have it supported transparently. CC @vsevel

lordofthejars commented 4 years ago

Yes this is my plan but first I need the health check PR merged to avoid having some merge conflicts and also to reuse some classes there.

sberyozkin commented 4 years ago

Hi Alex, @lordofthejars, I'd like to ask for some clarifications, so the current approach in the Vault extension is to acquire a token transparently by the extension (we've agreed with Vincent the user code itself does not do it), once the authentication is done. So I have to admit I'm not really sure how such a token can be stolen, unless it is intercepted, in which case the authentication channel is also compromised. Please note I'm all for the better Vault security, but we should have a clear message in mind for the users why a given feature is needed. CC @vsevel

lordofthejars commented 4 years ago

@sberyozkin what is compromised is not the channel itself (network) but the process of sharing the token (for example suppose you are setting this token manually in your server). Notice that since the token is wrapped initially, the operator who generates the token doesn't know the real token so it adds a layer of security. Of course, maybe this operator is the admin of Vault, then it has not much sense but aif not then it is a new layer of security.

Here there is a schema: https://d33wubrfki0l68.cloudfront.net/9e6751e39a64669cc963db6333e0fca3b4b130b8/640fa/img/vault-approle-workflow2.png

Or for example, someone pushes to git the code with the token, or stores the token in Kubernetes (and you don't want to use Kube auth) is not a secure way. So basically what you are attempting here is trying to minimize the blast in case the token is stolen. And this can happen during transmission or maybe when the application is running (someone accesses the server and checks the environment variable to get the token) or someone logs the token in log files. So not a silver bullet but a new layer of security.

lordofthejars commented 4 years ago

Also and generally speaking in security topic, in my experience (which I am not a super expert) is that there is no the right way to do the things, you just work as an onion, and some methods might work for you but the same method might not work to another project/company. So it is more about providing different choices (which of course they need to have sense) rather than saying this way is good or this is better as depending on the security team and the project everything can change. The idea for me is taht no Quarkus user can say that he cannot integrate with the Vault auth method because of no support.

sberyozkin commented 4 years ago

Hey @lordofthejars, re the process of losing the token in the process, that in itself makes sense, but can you confirm please that the token which the Vault extension is acquiring may not be dynamically allocated ? If it is generated on the fly, per the authentication session, then there will be no risk.

Right, but Vault has N engines and N auth mechanisms :-) I do agree we need to support the teams migrating to Quarkus but in some cases it may be better to see the actual requirement. Vault extension is already pretty complex so IMHO we need to grow it carefully.

What you've explained about the wrapped tokens is useful which is what I'm saying. we need to know what case will be supported. Please confirm that the Vault extension can indeed be exposed to this attack (i.e, the tokens may not always be tied to the authentication session)

Cheers

sberyozkin commented 4 years ago

@lordofthejars I've read about Vault token concepts and most of it makes sense, I'm just not really sure if a default userpass authentication method can get some token which came out of band. But lets continue the discussion :-)

lordofthejars commented 4 years ago

The attack is not in the Vault extension only, but it can happen everywhere, in the service, in the host, in the process of deploying, ...

Then with the current approach, no wrapped token, the valid token is set in configuration file, so anyone could get the token with access to that file (or because of an attacker get access to the service, repo, ....)

With wrapped the token you are setting is the wrapped one, so if it is compromised you detect it as someone already used the unwrap operation.

So for extension what we can do is I do the login with the wrapped token, then we unwrap the token. This process of unwrapping the token is done at the Vault server, not in service and it is returned to it after unwrapped. Then this second token is the valid one and is never stored at the disk, it is taken from the request of unwrap operation and used to login using the vault login process. So the user never saw this token nor an attacker.

Of course if someone breaks the https then you could sniff the traffic between service and Vault and take it, but this is something that is not a Quarkus Vault extension problem per se but more an infrastructure problem.

lordofthejars commented 4 years ago

well you can do user password to get access to unwrap the token, I mean that the auth is not token1 and then unwrap token. You can do u/p and unwrap token or kube auth + unwrap token.

The problem with user password is that they are a secret as well so if these secrets are compromised you loose everything. So with these two layers, an attacker might have the u/p but you'll detect when you try to unwrap the token.

sberyozkin commented 4 years ago

@lordofthejars Sure, what I'm asking is what is the origin of this token: https://github.com/quarkusio/quarkus/blob/master/extensions/vault/runtime/src/main/java/io/quarkus/vault/runtime/VaultAuthManager.java#L46

I'm really not sure what you mean with the token set somewhere in the file... My understanding of the code is that Vault will generate a token when the client auhenticates Thanks

lordofthejars commented 4 years ago

@sberyozkin No, the token is generated by the operator but this is the wrapped token, the unwrapped token is created at the same time but not show.

Now if you look at vault extensions auth methods you see that there is user/password, token, ... then you need to set these parameters somewhere to login. It can be the application.properties, it can be a Kubernetes resource or whatever, but you need to set them. So the secrets are stored not in Vault but somewhere and if they are stolen all is compromised. With this approach, all can be compromised as well but the good thing is that you are going to detect it easily and quickly.

Notice that wrap tokens are similar to other auth methods we have there but it adds a new layer of security and more importantly allows you to detect that the system is compromised.

Have you seen and followed this tutorial? https://learn.hashicorp.com/vault/secrets-management/sm-cubbyhole

lordofthejars commented 4 years ago

I think that when you see the code, you'll see that it is pretty simple (or should a prior :))

vsevel commented 4 years ago

hello, I must admit I was not familiar with the cubbyhole. when I created the extension I provided 3 auth mechanims:

k8s
userpass (for humans)
and approle, which seemed to be targetting apps and workflows.

so the Chef, Jenkins trusted entities described in the cubbyhole doc would seem to fit into this category also, no? approle has ttl also on the generated token, but on the secret id as well. it is not clear to me at this point which one is better, or if they should be combined (your first diagram shows exactly that)? may be it is worth reaching out the vault community for guidance? I am not sure cubbyhole is actually an auth mechanim. it looks more like a way to provide an additional protection on a secret (eg: an approle secretid, a token, ...). in that sense this might be useful in the toolboxl.

lordofthejars commented 4 years ago

Yes the example I showed is what I described here. Basically I am thinking from the point of view of Quarkus app what API I should support to implement the learn example I shared and in this case is the unwrap operation.

El El jue, 19 mar 2020 a las 22:47, Vincent Sevel notifications@github.com escribió:

hello, I must admit I was not familiar with the cubbyhole. when I created the extension I provided 3 auth mechanims:

k8s

userpass (for humans)

and approle https://www.vaultproject.io/docs/auth/approle/, which seemed to be targetting apps and workflows.

so the Chef, Jenkins trusted entities described in the cubbyhole doc would seem to fit into this category also, no? approle has ttl also on the generated token, but on the secret id as well. it is not clear to me at this point which one is better, or if they should be combined (your first diagram shows exactly that)? may be it is worth reaching out the vault community https://groups.google.com/forum/#!forum/vault-tool for guidance? I am not sure cubbyhole is actually an auth mechanim. it looks more like a way to provide an additional protection on a secret (eg: an approle secretid, a token, ...). in that sense this might be useful in the toolboxl.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/quarkusio/quarkus/issues/7900#issuecomment-601432447, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALSMYK2EN47H772HV7VXLTRIKHG5ANCNFSM4LNCG73Q .

-- Enviat amb Gmail Mobile

lordofthejars commented 4 years ago

Cubbyhole is not an auth method but they use it internally for providing single access to wrapped token

El El jue, 19 mar 2020 a las 23:21, Alex Soto asotobu@gmail.com escribió:

Yes the example I showed is what I described here. Basically I am thinking from the point of view of Quarkus app what API I should support to implement the learn example I shared and in this case is the unwrap operation.

El El jue, 19 mar 2020 a las 22:47, Vincent Sevel < notifications@github.com> escribió:

hello, I must admit I was not familiar with the cubbyhole. when I created the extension I provided 3 auth mechanims:

k8s

userpass (for humans)

and approle https://www.vaultproject.io/docs/auth/approle/, which seemed to be targetting apps and workflows.

so the Chef, Jenkins trusted entities described in the cubbyhole doc would seem to fit into this category also, no? approle has ttl also on the generated token, but on the secret id as well. it is not clear to me at this point which one is better, or if they should be combined (your first diagram shows exactly that)? may be it is worth reaching out the vault community https://groups.google.com/forum/#!forum/vault-tool for guidance? I am not sure cubbyhole is actually an auth mechanim. it looks more like a way to provide an additional protection on a secret (eg: an approle secretid, a token, ...). in that sense this might be useful in the toolboxl.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/quarkusio/quarkus/issues/7900#issuecomment-601432447, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALSMYK2EN47H772HV7VXLTRIKHG5ANCNFSM4LNCG73Q .

-- Enviat amb Gmail Mobile

-- Enviat amb Gmail Mobile

sberyozkin commented 4 years ago

@lordofthejars Thanks for the link. I'm still looking for some clarity about the origin of the token with the userpass method. You keep saying token should be configured somewhere. But it is not configured anywhere as far as I can see :-), can you please also point to some resource explaining the origin of the token in the userpath method, is it set somewhere manually or generated with ttl ?

May be you can indeed do a quick draft PR to clarify what exactly is being provided

lordofthejars commented 4 years ago

When the PR of health metrics is merged I'll do it as I need some bits from there. Also if you want we can do a bluejeans session and I explain everything. As you wish :)

sberyozkin commented 4 years ago

Hey @lordofthejars I certainly don't want to make adding a useful feature difficult :-). All I'm looking for is for some clarity: Is the userpass method at risk to return a stolen token to Quarkus given that Quarkus itself does not configure tokens. This is the 1st question and it would help us understand things better. If userpass methos is at risk of getting a stolen token (we are not talking about the secrets being stolen as the assumption is the auth channel is secure) then it would be a good reason to add an explicit support for the feature. The other question is along the lines of what Vincent has asked about, does cubbyhole complement or provides an alternative to either of the already supported methods. If it provides a viable alternative then it would be another good reason to support this feature. Etc. So that when a user asks why, I and others could answer clearly :-) Lets keep talking here for everyone to follow the discussion, Thanks, I'm positive something useful can be added to the extension as a result of this issue :-). The other thing is that Vincent's approval is what will let me merge eventually :-)

lordofthejars commented 4 years ago

Ok here are different things, the token that is returned by using user and password, in theory, cannot be compromised as communication happens at TLS level. What can be stolen as you said is the user or the password not in the transport but where they are originally stored, notice that they are not stored inside Vault, because you need them to access Vault, so they are in configuration files, environment properties, ... Well this is the basic problem of secrets in deployments, who have the secret, where it is stored, how it is loaded, ...

So with this approach, you are mitigating a bit the risk, not in the sense of stealing them, but to detect that they have been stolen and apply some measures. Notice that you could then implement "2 phase authentication, 3 phase authentication, ..."

Cubbyhole is not an auth method it is secret management, like the secrets one or the TOTP. The thing is that Vault uses internally cubbyhole for this auth method and they named Cubbyhole wrapped token, hence it might leave us to the confusion, but from the point of view of this issue, it is not about cubbyhole but as unwrapping tokens, just unwrapping tokens and re-logging with the unwrapped token, nothing more. No cubbyhole support nothing as for this auth method cubbyhole is something used by Vault and not for us.

So what we want to support is the next lifecycle:

I log to Vault with the authentication method you provide (it can be a token, user pass, approle or kube)
We ask Vault to unwrap the wrapped token provided in config file (notice that this token can only be used once).
In case cannot be unwrapped send an error as it means something is happening on your system.
If valid we relog with the unwrapped token.

Act as usual to inject secrets.

Then try to forget anything related to cubbyhole as this issue is not about cubbyhole. The only operation we need to implement is: curl --header "X-Vault-Token: s.3Kf3Xfn58Asr3bSDkRXATHrw" \ --request POST \ http://127.0.0.1:8200/v1/sys/wrapping/unwrap

That's it. So it is really simple just a POST method and a relogging, notice that no cubbyhole, and with just this, you are making your service more secure as you can detect possible stolen of auth secrets to authenticate against Vault.

If you want to do a video call for better understanding I have no problem.

sberyozkin commented 4 years ago

Hi @lordofthejars thanks, it is getting clearer, thanks for your patience :-). So you say there, I log to Vault with the authentication method you provide (it can be a token, user pass... and We ask Vault to unwrap the wrapped token provided in config file...

So how does it align if a user pass is used ?

userpass method returns a token to Quarkus
and we have a token set in the properties, what is next for these 2 tokens, which one is passed with that X-Vault-Token ?

lordofthejars commented 4 years ago

Ok then you do a login with user password, Vault returns a token that allows you to do some operations, in this concrete case this token only allows you to unwrap a token.

So:

user password returns a token with limited scope (token1)
Uses this token (token1) to unwrap a configured token (token2) in application.properties.
When token2 is unwrapped it returns a token3.
token3 is used to log in again to Vault and then you've got permission to get secrets.

Then token3 is a one used token, this means that if an attacker gets access to username, password and token2 they will be able to get secrets it is true, but as if you only used user and password, but the big difference is that using this approach the application will notice that this has happened so you'll be able to react quickly and investigate.

sberyozkin commented 4 years ago

@lordofthejars Excellent, now I see how it works :-) Thanks. So the flow then, if this special token is set then the above process follows. +1 CC @vsevel

sberyozkin commented 4 years ago

@lordofthejars It does appear to be a rather slow process though, 3 round trips to log in, but it will only be done once and will be optional, so for the sensitive deployments it can make sense

lordofthejars commented 4 years ago

Exactly depending on how the paranoic security team is, you can use it or not. Sorry to not be able to explained you correctly at first.

sberyozkin commented 4 years ago

@lordofthejars Np at all, thanks :-)

vsevel commented 4 years ago

Ok then you do a login with user password, Vault returns a token that allows you to do some operations, in this concrete case this token only allows you to unwrap a token.

thanks for the explanation.

using this approach the application will notice that this has happened

how? will the previously delivered token3 stop working?

lordofthejars commented 4 years ago

When you are trying to unwrap the token2 instead of receiving the token3 it returns an error message saying that the token has been already used.

vsevel commented 4 years ago

but then how does it play out in a microservice world, which usually share the same config. If I launch 3 replicas. are we saying that the first one will succeed, but the other 2 will be rejected because the token has already been used? even if I have just one replica and it dies and k8s restarts it. will it fail authenticating upon restart? I guess I am missing something.

lordofthejars commented 4 years ago

You'll have one token for each replica set as a config source. But in any case, notice that this method is more for none Kubernetes environments.

sberyozkin commented 4 years ago

Losing the secrets even once is sufficient though to get them used somewhere, and this method does not prevent it in itself....

lordofthejars commented 4 years ago

As I said in the comments this method does not prevent that an attacker can access to the system, what this method provides in contrast of another method, for example user password, is to detect that the token has been used in the wrong way. With user password one can use the user password and you'll never know a prior if it is a good access or not. With unwrap tokens you know that someone not valid is using it (and you detect it really fast) so the only thing you do is blocking the access.

Notice that it is much better this than just take some hours until you detect the misuse of the token.

sberyozkin commented 4 years ago

@lordofthejars Yes, this is true. While I understand the mechanics of this mechanism now, what we are saying then is that unless this mechanism is used, the userpass mechanism is insecure out of the box, since it will be at risk of the user pass being stolen and hence the token returned in

1. user password returns a token with limited scope (token1)

also being stolen. Which can really only be the case if we want to talk to Vault over an open HTTP outside of any barriers.

I'm not really convinced, but if, as discussed above, it can help some security teams have a better peace of mind, then sure, I'll keep my earlier +1

lordofthejars commented 4 years ago

Token can be stolen not just because of http but also because the repo where you are storing user passwords are compromised, security breach in your company, a malicious employee, a push of the wrong file to git repo, ... so it is not about http it https , it is about the whole security process and wrap tokens improve to detect any of these situations. I worked in the past with Vault in an electronic voting system and well security is really paranoic there, so I still remember some of the best practices, and in case of Vault this was one of used as even thought the original user password was travelling with a physical usb driver, this could be stolen, lost, key sniffed... So https is just one thing but there are a lot of things that are happening before that.

El El mar, 24 mar 2020 a las 12:23, sberyozkin notifications@github.com escribió:

@lordofthejars https://github.com/lordofthejars Yes, this is true. While I understand the mechanics of this mechanism now, what we are saying then is that unless this mechanism is used, the userpass mechanism is insecure out of the box, since it will be at risk of the user pass being stolen and hence the token returned in

user password returns a token with limited scope (token1)

also being stolen. Which can really only be the case if we want to talk to Vault over an open HTTP outside of any barriers.

I'm not really convinced, but if, as discussed above, it can help some security teams have a better peace of mind, then sure, I'll keep my earlier +1

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/quarkusio/quarkus/issues/7900#issuecomment-603182642, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALSMYJMVBXEAIRAFI6Y243RJCJZHANCNFSM4LNCG73Q .

-- Enviat amb Gmail Mobile

sberyozkin commented 4 years ago

@lordofthejars You know this space well. If with Vault one can get secrets lost somewhere before userpass is even used then indeed having an extra feature to have it detected early would be useful

vsevel commented 4 years ago

I spent some time trying to figure out the details. Probably nothing new for you @lordofthejars but I had to run through a complete example to feel better about it. I looked at Cubbyhole Response Wrapping, which provided the base, then AppRole Pull Authentication for a scenario that would be closer to a quarkus app.

here is the complete scenario: authenticating an app through approle, but instead of providing the app with the role and secret ids, we provide instead the role id and a short lived token that will allow the app get the secret id

admin persona:

docker run --rm --cap-add=IPC_LOCK -e VAULT_ADDR=http://localhost:8200 -p 8200:8200 -d --name=dev-vault vault:1.2.3
docker logs dev-vault 
# => <root_token>
docker exec -it dev-vault sh

export VAULT_TOKEN=<root_token>

cat <<EOF | vault policy write myapp -
path "secret/data/myapp" {
  capabilities = [ "read" ]
}
EOF

vault kv put secret/myapp password='pa$$w0rd'

vault auth enable approle
vault write auth/approle/role/myapp token_policies="myapp" token_ttl=1h token_max_ttl=4h
vault read auth/approle/role/myapp/role-id 
# => <role_id>

# instead of doing:
vault write -f auth/approle/role/jenkins/secret-id 
# => <secret_id>
# do:
vault write -wrap-ttl=60s -f auth/approle/role/myapp/secret-id 
=> <wrapping_token>

MyApp persona:

docker exec -it dev-vault sh

vault unwrap <wrapping_token> 
# => <secret_id>
vault write auth/approle/login role_id="<role_id>" secret_id="<secret_id>" 
# => <client_token>
VAULT_TOKEN=<client_token> vault kv get secret/myapp
# => the password

vault unwrap <wrapping_token> 
# => ERROR

this approach would work with other auth alternatives where you provide a wrapped token to the app instead of the auth secret (secret-id for approle, password for userpass, ...)

if we wanted to apply this to quarkus, this would mean that in the config, instead of having:

quarkus.vault.authentication.app-role.role-id=026457df-98c4-144e-a890-931419da8efb
quarkus.vault.authentication.app-role.secret-id=5caa3aa3-f8d5-1c94-9870-4635f7d4f345

we would have:

quarkus.vault.authentication.app-role.role-id=026457df-98c4-144e-a890-931419da8efb
quarkus.vault.authentication.app-role.secret-id-wrapped=s.X7byfHSbkTc08YgrtlV62HQp

and in the code we would do something like this:

        } else if (type == APPROLE) {
            VaultAppRoleAuthenticationConfig appRoleConfig = serverConfig.authentication.appRole;
            String roleId = appRoleConfig.roleId.get();
            String secretId = appRoleConfig.secretIdWrapped.isPresent()
                ? vaultClient.unwrap(appRoleConfig.secretIdWrapped.get()).get("secret_id")
                : appRoleConfig.secretId.get();
            auth = vaultClient.loginAppRole(roleId, secretId).auth;

and we would have to do something similar for all auth alternatives that have a secret information to hide (which is not the case for k8s).

if we look at the big picture, we are talking about quarkus applications that would:

get a wrapped token injected somehow in their config - an application.properties on disk?
would not be able to restart, unless a new wrapped token gets injected again

it is reasonable to think that this approach does not work for serverless or microservices, where a fundamental property is to be able to die and restart at will. from that perspective, and based on the original goal of quarkus, it does not seem that this feature would be expected. however I suppose quarkus aims at a lot more than just microservices, when I see discussions on CLI or batch processing. so may be there is a space where we want a quarkus application to start only after a config has been provisioned for it. for instance a job scheduler could be the trusted entity that can generate the short lived wrapped token, and launch a job with that token as a parameter.

if I am not entirely convinced that there is a big need for this at this point, the good news is that supporting it would not add any complexity, and can be implemented without breaking the current design (if I understood it correctly).

did I capture your idea correctly?

lordofthejars commented 4 years ago

Yes that's what I feel as well. Somewhere I said that this is not for Kubernetes but for normal services deployed i the old way. And as you said it is really simple to implement, so it is a "win" for Vault extension to support this and affecting almost 0 in the code.

vsevel commented 4 years ago

normal services deployed i the old way

even there I do not see a use case. even (non k8s) static workloads work with a config on disk that is not changed by a trusted entity every time we need to restart the service. we are running jboss servers at scale the old way, and the app config may change only when there is an official change request approved by the ops people. if the service crashes, it will restart with the exact same config it started previously. an operational reason for this is that the business service is critical, but the provisioning tool is not: if the provisioning tool is dead for 4 hours, it just delays new change requests to be applied. if the business service is dead for 4 hours that is another matter. putting a constraint on the provisioning tool to be there to inject a new wrapped token in the config of a critical app upon restart, means that the provisioning tool becomes critical as well. and we do not want that. at least that is my experience.

so I am not convinced by the applicability of this feature in general, meaning use cases exist but they are rare I think (and for k8s it is irrelevant).

but if you strongly believe there is a case for that feature, I will happily help any way I can. ;-)

lordofthejars commented 4 years ago

String secretId = appRoleConfig.secretIdWrapped.isPresent()
                ? vaultClient.unwrap(appRoleConfig.secretIdWrapped.get()).get("secret_id")
                : appRoleConfig.secretId.get();
            auth = vaultClient.loginAppRole(roleId, secretId).auth;

Notice that this is not how the example on Vault site is done. There they use username password, to get the wrapped token and they unwrap the token and the they log with that token.

lordofthejars commented 4 years ago

So my plan was not doing in this way, but using the same approach as in docs, so I do a username and password login then I do the unwrap and then I relogin with the unwrapped token, this is how the Vault documentation example works.

vsevel commented 4 years ago

the scenario I documented in https://github.com/quarkusio/quarkus/issues/7900#issuecomment-603981580 is strictly based on AppRole Pull Authentication from the documentation with the Response Wrap the SecretID twist, which you referred to in your https://github.com/quarkusio/quarkus/issues/7900#issuecomment-600982178.

the documentation explains the approach very well:

The RoleID is equivalent to a username, and SecretID is the corresponding password. ... SecretID is like a password. To keep the SecretID confidential, use response wrapping so that only the expecting client can unwrap the SecretID.

So RoleID can be part of the config. But instead of putting the SecretID in the config as well, you wrap it, and add the wrapping token in the config:

# instead of doing:
vault write -f auth/approle/role/jenkins/secret-id 
# => <secret_id>
# do:
vault write -wrap-ttl=60s -f auth/approle/role/myapp/secret-id 
=> <wrapping_token>

when you access vault using a client token, it is the same idea: instead of providing the client token to the client, you wrap it and provide the wrapping token. when the client wants to access vault, it will unwrap the wrapping token, find a client token in the response, and use that client token to access vault and start fetching secrets.

you always wrap the most secret information:

approle: it is the secret-id
userpass: it is the password
client token: it is the client token itself

I think that is exactly what SB is doing in Cubbyhole authentication: they create a token, wrap it, and add the wrapping token into the config, although I do not think their doc is accurate:

vault token-create -wrap-ttl="10m" command does not work
they do not specify a policy that the created token would have access to

I adapted my earlier complete example to show how we would do it for client token access with the 2 alternate methods with approle auth and with client token:

admin persona:

docker run --rm --cap-add=IPC_LOCK -e VAULT_ADDR=http://localhost:8200 -p 8200:8200 -d --name=dev-vault vault:1.2.3
docker logs dev-vault
# => <root_token>
docker exec -it dev-vault sh

export VAULT_TOKEN=<root_token>

cat <<EOF | vault policy write myapp -
path "secret/data/myapp" {
  capabilities = [ "read" ]
}
EOF

vault kv put secret/myapp password='pa$$w0rd'

##################### with approle auth ##############
vault auth enable approle
vault write auth/approle/role/myapp token_policies="myapp" token_ttl=1h token_max_ttl=4h
vault read auth/approle/role/myapp/role-id
# => <role_id>

# instead of doing:
vault write -f auth/approle/role/myapp/secret-id
# => <secret_id>
# do:
vault write -wrap-ttl=60s -f auth/approle/role/myapp/secret-id
# => <wrapping_token>
#####################################################

##################### with client token #############
# instead of doing:
vault token create -policy=myapp
# => token
# do:
vault token create -wrap-ttl=60s -policy=myapp
# => <wrapping_token>
#####################################################

MyApp persona:

docker exec -it dev-vault sh

##################### with approle auth ##############
vault unwrap <wrapping_token>
# => <secret_id>
vault write auth/approle/login role_id="<role_id>" secret_id="<secret_id>"
# => <client_token>
#####################################################

##################### with client token #############
vault unwrap <wrapping_token>
# => <client_token>
#####################################################

VAULT_TOKEN=<client_token> vault kv get secret/myapp
# => pa$$w0rd

vault unwrap <wrapping_token>
# => ERROR

the app persona does not start logging in. it starts by unwrapping the wrapping token, and finds in there the secret information that it is missing to either log in (e.g. with approle, and userpass) or directly access vault (e.g. client token).

SB has decided to limit response wrapping to the client token access. but response wrapping can be applied wherever you do not want to expose the secret information directly, as stated in the sm cubbyhole solution:

Any Vault response can be distributed using the response wrapping

back to quarkus, I think the proper way to present it to the user is by providing those additional properties:

quarkus.vault.authentication.app-role.secret-id-wrapped
quarkus.vault.authentication.userpass.password-wrapped
quarkus.vault.authentication.client-token-wrapped

what do you think?

vsevel commented 4 years ago

So my plan was not doing in this way, but using the same approach as in docs

I think there is an issue with the doc for the app persona (specifically on First, create a token with default policy.), or at least I could not make it work (and I do not understand why it is needed, or should that be something done by the admin persona instead?).

for the admin person it is ok:

docker run --rm --cap-add=IPC_LOCK -e VAULT_ADDR=http://localhost:8200 -p 8200:8200 -d --name=dev-vault vault:1.2.3
docker logs dev-vault
# => <root_token>
docker exec -it dev-vault sh

export VAULT_TOKEN=<root_token>

cat <<EOF | vault policy write apps -
path "secret/data/dev" {
  capabilities = [ "read" ]
}
EOF

vault kv put secret/dev username="webapp" password="my-long-password"
vault token create -policy=apps -wrap-ttl=120
# => <wrapping_token>

but in the app persona I get an error when creating a token for policy default:

/ # vault token create -policy=default
Error creating token: Error making API request.

URL: POST http://localhost:8200/v1/auth/token/create
Code: 403. Errors:

* 1 error occurred:
        * permission denied

and it is not required anyway to be able to unwrap the wrapping token. the only thing I could think of was that this should be the admin that creates the token for the default policy, and the app persona has to log in with this default token to be able to unwrap the wrapping token. but again this does not seem necessary because I have been able to unwrap a wrapping token with just VAULT_TOKEN=<wrapping_token> vault unwrap and no previous login.

here is to me the valid scenario for the app persona:

MyApp persona:

docker exec -it dev-vault sh

VAULT_TOKEN=<wrapping_token> vault unwrap
# => <client_token>

VAULT_TOKEN=<client_token> vault kv get secret/dev
# => my-long-password

and that is exactly what I was describing in my previous https://github.com/quarkusio/quarkus/issues/7900#issuecomment-605608332 for the client token access.

another way to verify that anybody can unwrap a wrapping token with no previously established token, is to do the unwrap with a curl:

> curl --header "X-Vault-Token: s.u4Nd4lV7R3Y4OvmrNWCeMRm7" --request POST http://127.0.0.1:8200/v1/sys/wrapping/unwrap | jq
{
  "request_id": "70389919-c9c6-cafd-d703-625ce704b3cd",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": null,
  "wrap_info": null,
  "warnings": null,
  "auth": {
    "client_token": "s.s6JKz1bedFGfAzbb8M8ZMBJG",
    "accessor": "bdhzFL0ZfLbejdJbF6Xq6KTp",
    "policies": [
      "apps",
      "default"
    ],
    "token_policies": [
      "apps",
      "default"
    ],
    "metadata": null,
    "lease_duration": 2764800,
    "renewable": true,
    "entity_id": "",
    "token_type": "service",
    "orphan": false
  }
}

vsevel commented 4 years ago

@lordofthejars

I ended up drafting a proposition based on discussions in this issue https://github.com/quarkusio/quarkus/issues/7900 and associated PR https://github.com/quarkusio/quarkus/pull/8248.

no PR at this point. everything is in: https://github.com/vsevel/quarkus/commit/ca68d6f1f45eff7caca6caf148fca10b81131eba

it has a complete implementation including tests for response wrapping applied to

client token auth and
secret id in approle auth

let me know if you want to take it from there, or if you would rather have me push it forward in a new PR.

cc @sberyozkin

lordofthejars commented 4 years ago

In my opinion is that we can close this one and send a new PR.

vsevel commented 4 years ago

ok, so I created https://github.com/quarkusio/quarkus/pull/8411 please feel free to review I have not done userpass, but that should be very similar

quarkusio / quarkus

Add support for Vault Unwrap Token #7900