treasure-data / td-client-java

Java Client Library for Treasure Data
http://docs.treasuredata.com/articles/java
Apache License 2.0
12 stars 22 forks source link

Support jittering backoff strategies #169

Closed yuokada closed 3 years ago

yuokada commented 3 years ago

Exponential Backoff And Jitter | AWS Architecture Blog

According to the blog entry above, jittering backoffs are more effective than exponential backoff in view of retry count and completion time.


miniway commented 3 years ago

@yuokada jittering backoffs looks good to me. Then how we configure or switch to the jittering ?

yuokada commented 3 years ago

@miniway In our presto? I don't consider yet. But, for current user using ExponentialBackOff can easily switch to jittering backoffs since all backoffs extend BackOff abstract class.

yuokada commented 3 years ago

Ah... you mean that I need to improve the class below to support jittering backoffs. Right? https://github.com/treasure-data/td-client-java/blob/master/src/main/java/com/treasuredata/client/TDClientConfig.java

If so, I didn't completely notice.

xerial commented 3 years ago

TDClientBuilder is an entry point for configuring TDClient https://github.com/treasure-data/td-client-java/blob/master/src/main/java/com/treasuredata/client/AbstractTDClientBuilder.java

We already have parameters (max retry, multipliers, etc.) for retyring so we should be able to provide a custom Backoff factory (default will be generating FullJittering class). For example, create an interface like this:

interface BackoffFactory {
   Backoff newBackoff(TDClientConfig config); 
}

Then, support setting a custom BackoffFactory at TDClientBuilder.

We also need to change this part https://github.com/treasure-data/td-client-java/blob/c02b9bbd2d94e2b948cf6bde5f07448ca2f1a4a1/src/main/java/com/treasuredata/client/TDHttpClient.java#L345