Prune inactive entities

[g-psantos]

TL;DR

We're considering a solution that will create a significant number of Entities in Vault's Identity backend. There appears to be no built-in functionality to (a) automatically prune Entities that have been inactive for some time or (b) determine when an Entity was last active. This means that the Identity backend would continue to grow unchecked over time.

Ideally, we'd like the ability to configure the Identity backend to automatically prune inactive Entities after "X" days. At a minimum, we'd like the ability to determine when each Entity was last active (so that we can script querying and deleting Entities).

The Long Form

Is your feature request related to a problem? Please describe.

We're looking to issue TLS client certificates, using Vault's PKI backend, to machines in our infrastructure for mutual TLS authentication to a service. Ideally, the client certificates should contain an attribute that uniquely identifies the client. This then enables the service to positively identify each client and accurately record audit log entries. On AWS, the approach we're contemplating is as follows:

1. Configure clients to authenticate to Vault using the AWS authentication backend in the EC2 mode.
2. Configure the AWS authentication backend to base Entity Aliases on instance_id for EC2 authentications.
3. Create an ACL policy that points to the PKI role and limits the allowed values for the common_name parameter to identity.entity.aliases.<mount accessor>.name, where "mount accessor" refers to the AWS authentication backend.

The problem with this approach is that Vault will create an Entity and Entity Alias for each new EC2 instance that needs to interact with the service. With horizontal autoscaling, this could quickly bog down Vault.

Describe the solution you'd like

The Identity engine should be configurable to automatically prune Entities that have been inactive for a set "X" number of days. Alternatively, there should be a record of when each Entity last authenticated to Vault that is returned with the Read Entity API endpoints.

Describe alternatives you've considered

1. Ignore it and hope we can cope with all the Entities/Aliases long term.
2. Keep the AWS backend's aliases associated with Vault Role IDs instead of Instance IDs. The trade-off is that clients would need permission to specify arbitrary Common Names, possibly impersonating other clients. Nevertheless, we'd still have some degree of client authentication (clients still need access to Vault), and it's likely that impersonation attacks could at least be traced through Vault's own audit logs.

Explain any additional use-cases

While the use case we have in mind is very specific, the ability to clean up inactive Entities is likely useful to have in general.

~Additional context~

[heatherezell]

Are you looking for something like vault tidy? https://developer.hashicorp.com/vault/api-docs/secret/pki#tidy

[g-psantos]

It'd be something like that but to clean up inactive Entities from the Identity engine.

(The whole story about the PKI backend is there only to provide context as to how we'd end up with too many Entities in the Identity engine -- the TL;DR at the top is really where the request is described succinctly. In retrospect, it might've been better to leave out all that context :shrug: )

[heatherezell]

Got it! Thanks for the clarification. :)

[alexisdondon]

It would be nice to have vault let configure this for example at authentication engine mechanism let say for example when you configure auth engine have a ttl on entities.

I've got such a use case in kubernetes authenticating service accounts within namespace.

[MaxDiOrio]

This would be nice - it appears that auth_kubernetes just keeps right on creating entities/aliases. I have close to 100k entities now.

[shanewhite97]

WE have the same issue using k8s, is this something that could be coming soon?

[getsomebread]

Since upgrading to Vault 1.15.5, have had two situations where Vault worked fine from web UI, but API access with approles were failing. Fix in both cases was to delete the many entities we observed. This feature would help us greatly.