search

mittwald / kubernetes-secret-generator

Kubernetes controller for automatically generating and updating secrets
Apache License 2.0
330 stars 57 forks source link

The secret-generator pod occasionally fails to start after the pod is being deleted #40

Closed eranreshef closed 3 years ago

eranreshef commented 3 years ago

Describe the bug The secret-generator container sometime restarts after the pod is being replaced.

To Reproduce

  1. Deploy the secret-generator with the manifests supplied in this repo.
  2. After the pod is up, delete it.
  3. The new single container in the new pod will sometime restart due to the wait period when it tries to be the leader.

Expected behavior A new pod will start and be ready immediately.

Environment:

Additional context The generator logs:

kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644831.4539402,"logger":"cmd","msg":"Operator Version: 0.0.1"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644831.4539738,"logger":"cmd","msg":"Go Version: go1.15.5"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644831.4539793,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644831.453984,"logger":"cmd","msg":"Version of operator-sdk: v0.16.0"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644831.4541202,"logger":"leader","msg":"Trying to become the leader."}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644832.172396,"logger":"leader","msg":"Found existing lock","LockOwner":"kubernetes-secret-generator-68d6f67df7-bs2lj"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644832.1823065,"logger":"leader","msg":"Not the leader. Waiting."}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644833.3122263,"logger":"leader","msg":"Not the leader. Waiting."}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644835.699852,"logger":"leader","msg":"Not the leader. Waiting."}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644840.244563,"logger":"leader","msg":"Not the leader. Waiting."}
- kubernetes-secret-generator-68d6f67df7-dghwg › kubernetes-secret-generator
+ kubernetes-secret-generator-68d6f67df7-dghwg › kubernetes-secret-generator
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644841.9950554,"logger":"cmd","msg":"Operator Version: 0.0.1"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644841.9950805,"logger":"cmd","msg":"Go Version: go1.15.5"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644841.9950848,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644841.9950886,"logger":"cmd","msg":"Version of operator-sdk: v0.16.0"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644841.9952486,"logger":"leader","msg":"Trying to become the leader."}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644842.7113423,"logger":"leader","msg":"Found existing lock","LockOwner":"kubernetes-secret-generator-68d6f67df7-bs2lj"}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644842.723332,"logger":"leader","msg":"Not the leader. Waiting."}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644843.855035,"logger":"leader","msg":"Leader pod has been deleted, waiting for garbage collection do remove the lock."}
kubernetes-secret-generator-68d6f67df7-dghwg kubernetes-secret-generator {"level":"info","ts":1617644846.2358556,"logger":"leader","msg":"Became the leader."}
mittwald-machine commented 3 years ago

There has not been any activity to this issue in the last 30 days. It will automatically be closed after 7 more days. Remove the stale label to prevent this.