contiv / netplugin

Container networking for various use cases
Apache License 2.0
512 stars 177 forks source link

The etcdv2 Watcher bug in the contiv. #1140

Open liucimin opened 6 years ago

liucimin commented 6 years ago

Description

Contiv as CNI plugin in Kubernetes. Contiv in using etcdv2 in the project. And the netplugin create some watcher to watch the etcd data.Such as state/nets. But if the network interrupt,and the etcd index has been cost larger than 1000.The watcher must fail because that the etcdv2's just save 1000 events before now index .

Then we use netctl to create a new network in netmaster,but the netplugin can not get the change. If we use the new network to create pod,it will fail!

Expected Behavior

Pod should be created by the new network.

Observed Behavior

Problem seems to be from the etcdv2's mechanism and the etcdv2 library.

Steps to Reproduce (for bugs)

1.Create a new network for the contiv. 2.Use the contiv until the etcd index 1000 larger than step 1. 3.Interrupt the network between netplugin and etcd. 4.Resume the etwork between netplugin and etcd. 5.Create a new network for the contiv. 6.Create new pods.

Your Environment

Server Version: Version: 1.2.1-46-g99b69e9-unsupported GitCommit: 99b69e9-unsupported BuildTime: 03-29-2018.01-58-52.UTC