jet / kafunk

Kafunk: F# Kafka client
https://jet.github.io/kafunk/
Other
160 stars 63 forks source link

Exponential retry policy #201

Closed eulerfx closed 6 years ago

eulerfx commented 6 years ago

This PR adds an exponential, randomized retry policy, and changes the defaults to match native client.

Native config: https://kafka.apache.org/documentation/#producerconfigs

The new retry policy is:

RetryPolicy.expRandLimitBoundedMs 
  50     // initial delay
  1.5     // exponential multiplier
  0.2    // randomization factor
  1000 // max delay
  2       // max attempts

And the new defaults are:

Connection Retry: RetryPolicy.expRandLimitBoundedMs 50 1.5 0.2 1000 2 Request Retry: RetryPolicy.expRandLimitBoundedMs 100 1.5 0.2 1000 2

Overall, these are more aggressive than what they previously were:

Connection Retry: RetryPolicy.constantBoundedMs 3000 2 Request Retry: RetryPolicy.constantBoundedMs 1000 2

There's also a bootstrap reconnect retry policy, governing attempts to find a bootstrap broker, which remains at RetryPolicy.constantBoundedMs 1000 3 as the default.