portworx / kvdb

Generic Key-Value interface
Apache License 2.0
40 stars 12 forks source link

Etcd v3: Create a separate client for Maintenance APIs. #52

Closed adityadani closed 6 years ago

adityadani commented 6 years ago

Currently the maintenance client is only used for the Status API, to fetch the endpoint status. However if the Status API errors out for an endpoint, the etcd client code marks the pinned address (which is reachable) as not reachable instead of the actual endpoint for which the Status command failed. This causes the etcd balancer to go into a retry loop trying to fix its healthy endpoints.

Etcd's maintenance APIs use a retryable client. Irrespective of the API being called this client removes the pinned address from etcd's balancer if any of the APIs fail. However the Status API itself takes an endpoint as an input and that can be different from the pinned address.

https://github.com/etcd-io/etcd/blob/v3.3.1/clientv3/retry.go#L102

This issue seems to be fixed in etcd's master branch, but could not find it in any released branches.