hpgrahsl / kafka-connect-mongodb

**Unofficial / Community** Kafka Connect MongoDB Sink Connector -> integrated 2019 into the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector
Apache License 2.0
153 stars 60 forks source link

[feature] exponential backoff for retries #61

Open hpgrahsl opened 5 years ago

hpgrahsl commented 5 years ago

The current behaviour of the connector w.r.t retries is a pretty naive one. It would make a lot of sense to implement an exponential backoff mechanism as a more robust retry policy. Thus all errors for which it is reasonable to have retries in the first place should respect new retry settings that allow for non-linear timeouts, ideally including randomized jitter. Also this new approach to deal with timeouts on retries can/should completely replace what's currently there.