search

kopeio / etcd-manager

operator for etcd: moved to https://github.com/kubernetes-sigs/etcdadm
Apache License 2.0
164 stars 45 forks source link

Backups not coming in after OpenStack token expires: Reauth bug? #337

Open kciredor opened 4 years ago

kciredor commented 4 years ago

Hi,

With reference to kops issue: https://github.com/kubernetes/kops/issues/9730

etcd-manager should be reauthenticating to OpenStack after the original token expires. I can see logic in the code to do so, but it does not seem te work.

I can see the controller loop coming by in the logs and creating backups of etcd to my OpenStack swift container. After 4 backups - which means 1 hour passed by - the backups stop coming in. This is exactly the timeout expiry value of my OpenStack provider.

Commit 8faecdad725d05f9c7375461cbf4f3dbbec6e527 made significant changes to reauth. I've tried the etcd-manager version before that commit as well: no luck, Authentication failed coming in as well.

Any thoughts?

kciredor commented 4 years ago

This suggested fix actually works great: https://github.com/kubernetes/kops/issues/9730#issuecomment-672157605