search

albertrdixon / romulus

A kubernetes ingress controller
MIT License
103 stars 12 forks source link

When romulus cannot connect to apiserver, it deletes everything already in etcd. #12

Closed DorianGray closed 8 years ago

DorianGray commented 9 years ago

I just saw this happen on our cluster. AWS killed our kubernetes apiserver yesterday and we forgot about it. https://github.com/timelinelabs/romulus/issues/3 hit and because of my patch romulus restarted. When romulus came up and couldn't connect to the apiserver, it deleted everything in etcd. I don't think this is expected behavior... I think that if romulus cannot talk to kubernetes for whatever reason, it should assume the data already in etcd is current and leave it alone until kubernetes can be talked to.

albertrdixon commented 9 years ago

heh, for sure. If romulus cannot connect to kubernetes it should:

a) leave everything in etcd alone b) simply try to reconnect every INTERVAL seconds (2s) until told to stop (killed)