search

hashicorp / vault

A tool for secrets management, encryption as a service, and privileged access management
https://www.vaultproject.io/
Other
31.03k stars 4.2k forks source link

Symlink style secrets #1284

Closed nickithewatt closed 6 years ago

nickithewatt commented 8 years ago

We quite regularly run into the situation where we have some information in Vault which is common to a few different teams, as well as some team specific info. In order to simplify management, we would ideally like to define ACLs etc to restrict access for a team to only access information under their team area, even if this ultimately means under the covers the secrets are common.

Take the following example below. Team 1 and Team 2 both need access to common com-secret1 and com-secret2. Team 1 additionally has its own versions of secret4 and secret5, whilst team2 has its own version of secret4 and secret6. 

/featurex/common/com-secret1
                 com-secret2
/featurex/team1/secret4
                secret5
/featurex/team2/secret4
                secret6

We could create ACLs granting specific access to the common area, and have team1 read secrets directly from the common area, however from a management perspective its simpler and easier to only have to deal with secrets in your own section, using your own names or aliases. Currently if we want this simplified management approach we would need to duplicate the secrets again under the team areas which is not ideal.

What would really be nice however is if we could rather have the concept of a "symlink" to the real secret. i.e.

/featurex/common/com-secret1
                 com-secret2
/featurex/team1/secret1* --> /featurex/common/com-secret1
                secret2* --> /featurex/common/com-secret2
                secret4
                secret5
/featurex/team1/secret1* --> /featurex/common/com-secret1
                secret2* --> /featurex/common/com-secret2
                secret4
                secret6

* = symlink style reference

This way all ACLs specified could simply be specific against team1 and team2, using whatever alias and or names makes sense for them, and under the covers this will then point at the common shared secrets - no duplication required.

This does obviously present some additional ACL considerations such as whether you should be allowed to "symlink" to an existing secret in the first place. Options in this area could include adding a "symlink" capability, i.e. team1 and team2 could have an ACL which permits symlinking (read only probably) to items under the /featurex/common/* path

This would ease the burden of having to make tradeoffs between simplified management and duplication of secrets.

Thanks Nicki

jefferai commented 6 years ago

This is quite complicated due to being specifically tied to a single backend type but at the same time would require custom logic in core to understand that it's referencing multiple paths and do ACL checks on both. We don't plan to implement this, so closing.

v6 commented 6 years ago

// ,

We could create ACLs granting specific access to the common area, and have team1 read secrets directly from the common area, however from a management perspective its simpler and easier to only have to deal with secrets in your own section, using your own names or aliases.

This would imply a layer of indirection for policies, I'd imagine, in that allowing access to a path that seems limited will actually end up affecting other teams, right?

whazor commented 5 years ago

maybe there could be a custom secret engine that only does symlinks?

Merlingilb commented 5 years ago

This is quite complicated due to being specifically tied to a single backend type but at the same time would require custom logic in core to understand that it's referencing multiple paths and do ACL checks on both. We don't plan to implement this, so closing.

@jefferai So what is your advise for this? What is the way to go to eliminate duplicate secrets? (Is dangerous in case of changing password)

pznamensky commented 5 years ago

We have lots of intersecting secrets between different teams and apps, that's why this issue is the only one which blocks using Vault in our environment.

Merlingilb commented 5 years ago

I cannot imagine that this isn‘t a feature many customors would want to use.

whazor commented 5 years ago

There is support for secret engine plugins, maybe the creation of a community vault secret engine plug-in would allow for namespacing, sharing secrets across teams, remote vaults, and other things I didn't think about.

On Thu, Aug 22, 2019, 18:13 Merlingilb notifications@github.com wrote:

I cannot imagine that this isn‘t a feature many customors would want to use.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hashicorp/vault/issues/1284?email_source=notifications&email_token=AABM65RNMBWKZC7WZ56KNATQF23LTA5CNFSM4B72FS5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD45TFTI#issuecomment-523973325, or mute the thread https://github.com/notifications/unsubscribe-auth/AABM65RIYN6GZ7PZ7EXIJQ3QF23LTANCNFSM4B72FS5A .

Merlingilb commented 5 years ago

It seems like hashicorp has no answer or solution to this

LowzG commented 5 years ago

+1 this would be a very useful feature to have

v6 commented 5 years ago

I respect the design decision to shy away from this.

I have some experience with Policy debugging.

And although I'm sure there are plenty of responsible ways to use it, the foot-gun potential with this is also huge, because of the increased side effects of changes to a Vault policy.

If you have a scenario where multiple teams need access to the same secret, it sounds like you might consider organizing your secrets better.

E.g. if I have

team1/db/passphrase

and

team2/db/passphrase

wouldn't it be easier to simply create a db/passphrase path, then create a policy for team1 and team2 that includes that db/passphrase path's access?

That's what I mean when I say that the apparent need for symlinks may reflect an organizational shortcoming.

v6 commented 5 years ago

@Merlingilb does "no" count as an answer: https://github.com/hashicorp/vault/issues/1284#issuecomment-373201034

pznamensky commented 5 years ago

E.g. if I have

team1/db/passphrase

and

team2/db/passphrase

wouldn't it be easier to simply create a db/passphrase path, then create a policy for team1 and team2 that includes that db/passphrase path's access?

@v6 thanks for the explanation. But let's guess you have hundreds of teams (applications) and more than one shared secret. And that shared secrets are not common for all, but one set of secrets should be available only for one set of apps, another set of secrets for another set of apps and so on. In these setups, it's incredibly hard to manage policies and paths with shared secrets.

v6 commented 5 years ago

Why would this be difficult to scale up?

InfoSec teams deal with PAM for shared vs team services all the time.

Perhaps an example would help me understand what you mean.

pznamensky commented 5 years ago

@v6 We have the same scenario as in the first message on this issue. Let's imagine that we have hundreds of applications and the following structure in Vault:

/app1/secret1
      secret2
      secret3

/app2/secret4
      secret5
...
/appN/secretM
      secretM+1

common-secret1 should be available for app1...app100 common-secret2 should be available for app20...app120 common-secret3 should be available for app80...app90

at the moment we have only two options:

  1. Add additional policies for each app which use shared secret It's not convenient and rarely leads to problems with renaming secrets because those paths are already used in apps config
  2. Copy shared secret inside each app In this case, you can't update a secret at the same time for several applications, and you can forget to change it for some apps.

But in "symlink-style" secrets, you could create "symlinks" to shared secrets and have only one policy for each application which is straightforward and convenient.

jefferai commented 5 years ago 
common-secret1 should be available for app1...app100
common-secret2 should be available for app20...app120
common-secret3 should be available for app80...app90

at the moment we have only two options:

Add additional policies for each app which use shared secret
It's not convenient and rarely leads to problems with renaming secrets because those paths are already used in apps config

It may not be convenient to have many policies, but it also allows for precise segregation of access control, and if paths do change, you can update a single policy and have it take effect everywhere.

My guess is that you are relying on creating and defining tokens with policies attached. This is indeed very cumbersome. Instead I would suggest defining identity groups, attaching appropriate policies to the groups, and adding appropriate applications to appropriate groups.

The other nice thing about keeping each set of distinct permissions to a single policy is that it is very declarative and this makes it relatively straightforward to manage policies in code. A lot of users use the TF provider or relatively simple scripts that simply iterate over policy files and ensure what's in Vault matches; this can be triggered by e.g. a PR being merged in a Git repository, so that you can have easy and open oversight for policy changes.

I would argue that for any large enough deployment (I talked to a customer recently who is onboarding 17000 applications to Vault) managing policies as code is a must, not optional.

Typositoire commented 4 years ago

For those still interested. I've built this to actually do symlinks in Vault. https://github.com/Typositoire/go-vln

v6 commented 4 years ago

I like the sequence diagram!

image

Although it does mean that there are now two systems involved in a given authorization check against a given path, maybe even 3 if the customer's using Sentinel.

For example, if I look at all the declarations of a given path in Vault policy contents, and see which roles have access to the Vault policies with those declarations, there still may be other roles that have a different path declared which also have access to that.

There are a few more chances for side effects there.

Maybe the facade could also require some kind of special comment or something? Still seems like it's adding another layer of indirection to Vault policies.

IMO, the philosophy of IAM, especially for microservices and IoT, will move closer to the objects and capabilities themselves as first class citizens rather than making more complex and rich authorization objects. That means more "pieces" of authorization that are simpler and simpler, as well as composable. I think part of HashiCorp's success is partly due to the underlying elegance of its token auth method.

Doing composition in a robust way for microservices means pushing indirection and other complexity out of policy creation and into other areas. Some designs in the past have taken this to an extreme, and achieved surprising results.

Composition of multiple access policies by their assignment (authorization) to an authentication class is a natural direction of this trend, rather than increasingly detailed policies.

If I make an access policy more closely reflect the subjects (identities) who get the policies, rather than the objects(various Vault API paths & operations) to which the policy gives access, it leads closer to the Unattenuated Composition (p. 23) problem in authority manipulation behavior, and defeats the purpose of the wonderfully granular design of Vault policies.

"Minimize the number of ACL policies" kind of defeats the point. Vault's authorization and authentication design has enough of that Unattenuated Composition stuff going on without intentionally making it worse.

Still overall for an application of the Facade pattern it seems pretty straightforward.

v6 commented 4 years ago

I probably sound waaaay more opinionated about this than I actually am. I just like thinking about this stuff I guess.

The only examples of Attenuated composition in Vault's auth+auth design are, I think of off the top of my head are at the namespace owner (or effective owner) level and at the token-generation level, because at no other levels of authority can I take my authorization and give someone else a useful reduced version of it. In the case of giving someone a Vault token that I've received from an authentication role I have, the only reduction in access the recipient would have compared with my access would be the time limit on the tokens lease, because I can use whatever proofs of ID that I have to re-auth to get another token, but the recipient of the token cannot.

sergevs commented 3 years ago

As I faced with the similar requirements to provide a "teams access" to particular secrets, I've created a simple application which may fulfill the missing feature. You may take a look on VaultGroupSecretsManager . Feel free to contact me if you have any questions.

sumanthkumarc commented 2 years ago

+1 for this feature. im not sure why anyone not require this feature, without duplicating secrets across paths.

whazor commented 2 years ago

The main problem I (and others) have is:

F-PMOR commented 1 year ago

+1 for creating a link

Mr-Aloof commented 8 months ago

+1 for this feature. The built-in native mechanism is very lacking. We have to use third-party hacks.

kk9090904565 commented 8 months ago

+1 This feature could make cicd jobs quite easily to use without unnecessary workarounds

SHA65536 commented 6 months ago

+1 This feature would really help us!

raphaeldavidf commented 4 months ago

+1 This feature would really help us!