spring-projects / spring-vault

Provides familiar Spring abstractions for HashiCorp Vault
https://spring.io/projects/spring-vault
Apache License 2.0
283 stars 186 forks source link

Secret rotation failing after LoginTokenExpiredEvent #867

Closed benallaerts closed 4 months ago

benallaerts commented 4 months ago

Hey,

Problem I'm trying to use spring-vault in my application with dynamic secret rotation for the database.

All secret renewals and rotations are working fine until the login token expires. On token expiration all secrets are correctly rotated, but on the next renewal the following exception is thrown

org.springframework.vault.VaultException: Cannot renew lease
    at org.springframework.vault.core.lease.SecretLeaseContainer.doRenewLease(SecretLeaseContainer.java:766)
    at org.springframework.vault.core.lease.SecretLeaseContainer.renewAndSchedule(SecretLeaseContainer.java:644)
    at org.springframework.vault.core.lease.SecretLeaseContainer.lambda$scheduleLeaseRenewal$4(SecretLeaseContainer.java:636)
    at org.springframework.vault.core.lease.SecretLeaseContainer$LeaseRenewalScheduler$1.run(SecretLeaseContainer.java:937)
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:96)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.NullPointerException: Cannot invoke "org.springframework.vault.core.lease.SecretLeaseContainer$LeaseRenewalScheduler.isLeaseRenewable(org.springframework.vault.core.lease.domain.Lease, org.springframework.vault.core.lease.domain.RequestedSecret)" because "renewalScheduler" is null
    at org.springframework.vault.core.lease.SecretLeaseContainer.doStart(SecretLeaseContainer.java:408)
    at org.springframework.vault.core.lease.SecretLeaseContainer.onLeaseExpired(SecretLeaseContainer.java:810)
    at org.springframework.vault.core.lease.SecretLeaseContainer.doRenewLease(SecretLeaseContainer.java:742)
    ... 11 more

The void restartSecrets() method in the SecretLeaseContainer seems to cancel and clear all stored renewals on login token expiration, but the newly created LeaseRenewalScheduler used to rotate the secret is never added to the stored renewals. This seem to cause the above exception when the next rotation is attempted.

Setup framework version
java 21
spring 6.1.6
spring-boot 3.2.5
spring-vault 3.1.1
Hashicorp Vault 1.16.3

Authentication is done using APPROLE credentials

mp911de commented 4 months ago

That's fixed now.

sebastianreloaded commented 2 months ago

Since this is a big Problem with long living services, is there an ETA for Release 3.1.2 ?