spring-cloud / spring-cloud-vault

Configuration Integration with HashiCorp Vault
http://cloud.spring.io/spring-cloud-vault/
Apache License 2.0
270 stars 151 forks source link

SecretLeaseContainer not aware of Vault Session token expiry #698

Closed sczachariah closed 9 months ago

sczachariah commented 10 months ago

Describe the bug

Hello,

We have implemented database dynamic credentials using spring cloud vault bootstrap properties and utilising VaultLeaseConfig similar to in here (https://secrets-as-a-service.com/posts/hashicorp-vault/rotate-dynamic-relational-database-connection-in-spring-at-runtime/#:~:text=To%20rotate%20the%20database%20credentials,to%20use%20the%20new%20credentials) for updating the hikari properties runtime when the dynamic credentials change.

All in all everything is working well and thank you for the amazing implementation and feature.

But we hit a snag every now and then when the session token expire.

So vault drops all the existing leases including database dynamic role lease, when the session token expire, irrespective of the ttl on dynamic role.

It seems SecretLeaseContainer is not aware of when the session token expire. As a result SecretLeaseExpired event is not fired and application goes into a fail state due to invalid db credentials, until the SecretLeaseContainer counts down on the ttl and fires next SecretLeaseExpiredEvent.

So we were wondering if it is possible for SecretLeaseContainer to be made aware of session token expiry by the LifecycleAwareSessionManager ?

Aplologies if this is handled in the latest version of the library. Just trying to find some pointers to resolve the issue.

mp911de commented 10 months ago

Thanks for reaching out.

Both components aren't aware of each other in the sense that the login token could expire, so we cannot communicate anything here.

I'm actually wondering why Vault drops all leases that were created using a particular Login Token. I just tried it out locally and was beyond surprised because it breaks a few assumptions.

Do you have a pointer to the docs that explain this behavior? I wasn't able to look it up right now.

Can you file a ticket at https://github.com/spring-projects/spring-vault/issues because Spring Vault carries all the infrastructure for Spring Cloud Vault?

You're right that SecretLeaseContainer needs to unconditionally send expiry events upon token rotation to request new secrets. After rotating the current secrets, it can then attempt revoking the previous secrets to run a form of cleanup.

sczachariah commented 9 months ago

Thank you very much @mp911de for your response.

Yes, we were equally surprised with the behaviour as this was not obvious.

Could find some wording along the lines in HashiCorp docs - https://developer.hashicorp.com/vault/docs/concepts/tokens#:~:text=After%20the%20current%20TTL%20is%20up%2C%20the%20token%20will%20no%20longer%20function%20%2D%2D%20it%2C%20and%20its%20associated%20leases%2C%20are%20revoked.

I will raise a ticket in Spring Vault as well.

mp911de commented 9 months ago

After the current TTL is up, the token will no longer function -- it, and its associated leases, are revoked

Thanks a lot. This is pretty essential and we have to fix it.

mp911de commented 9 months ago

This has been addressed in the latest 4.1.0-SNAPSHOT builds. Care to upgrade (also, make sure to use Spring Vault 3.1.0-SNAPSHOT) and test whether the fix works for you?

sczachariah commented 9 months ago

Amazing. Thank you very much @mp911de. Will test and let you know.