Retry tool

[GoogleCodeExporter]

Hello

I implement a retry tool which can retry task(wrapped in Callable) easily, 
whether it's valuable to be added in guava library?

Only 2 classes: Retry, RetryBuilder.

Example:

Retryer retryer = new RetryerBuilder()
   .times(3)
   .interval(10, SECONDS)
   .when(timeout())
   .build();

Response response = retryer.callWithRetry(new Callable<Response>() {
   public Response call() throws Exception {
      return removeService.upload(request);
   }
});

response.doSomething();
Retry condition:

private static Predicate<Exception> timeout() {
   return new Predicate<Exception>() {
      public boolean apply(Exception exception) {
         if (exception instanceof TimeoutException) {
            return exception.getMessage().startsWith("Retryable");
         }
         return false;
      }
   };
}

Project url : http://code.google.com/p/google-guava-retryer/

All java sources & test has been attached.

Original issue reported on code.google.com by Nert...@gmail.com on 3 Dec 2010 at 2:00

Attachments:

• sources.zip

[GoogleCodeExporter]

retryer.retry is more concise than retryer.callWithRetry :)

Original comment by Java2Ent...@gmail.com on 3 Dec 2010 at 2:14

[GoogleCodeExporter]

I refer to the interface TimeLimiter of guava when design the signature, the 
usage and structure of which is similar to Retryer, the method is named 
callWithTimeout, it is simple and clear.

Original comment by Nert...@gmail.com on 3 Dec 2010 at 2:28

[GoogleCodeExporter]

FYI, we are experimenting with one or two approaches internally at the moment.

Original comment by kevinb@google.com on 12 Jan 2011 at 10:37

• Changed state: Accepted
• Added labels: Type-Enhancement

[GoogleCodeExporter]

Original comment by kevinb@google.com on 13 Jul 2011 at 6:18

• Changed state: Triaged

[GoogleCodeExporter]

Original comment by kevinb@google.com on 16 Jul 2011 at 8:32

• Changed state: Accepted

[GoogleCodeExporter]

Update sources!

Original comment by Nert...@gmail.com on 18 Aug 2011 at 9:09

Attachments:

• sources.zip

[GoogleCodeExporter]

Issue 797 has been merged into this issue.

Original comment by cgdec...@gmail.com on 20 Nov 2011 at 4:30

[GoogleCodeExporter]

Original comment by fry@google.com on 10 Dec 2011 at 3:58

• Added labels: Package-Concurrent

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Sorry for the successive posts. My earlier design had serious issues, so I'm 
reposting it.

I had the same idea, and built a solution which is a bit more complex, but also 
more configurable. The differences are:
 - configurable stop strategy (number of times, delay, or custom)
 - configurable wait strategy (fixed delay, random delay, incrementing delay, or custom)
 - different exception handling

Usage example:

    Retryer<Response> retryer =
        RetryerBuilder.<Response>newBuilder()
                      .withStopStrategy(StopStrategies.stopAfterAttempt(4))
                      .withWaitStrategy(WaitStrategies.fixedWait(1L, TimeUnit.SECONDS))
                      .retryIfRuntimeException()
                      .retryIfExceptionOfType(IOException.class)
                      .retryIfResult(Predicates.<Response>isNull())
                      .build();
    try {
        return retryer.call(callable);
    }
    catch (ExecutionException e) {
        // the call threw a checked exception which didn't cause a retry
        // encapsulated in the execution exception
        Throwables.propagateIfPossible(e.getCause(), SomeCheckedException.class);
        throw new RuntimeException("unexpected", e.getCause());
    }
    catch (RetryException e) {
        // the retry was aborted because the call didn't succeed
        // it's also possible to get the last attempt result or exception
        // from the RetryException
        throw new RuntimeException("Call never succeeded", e);
    }

Original comment by jni...@gmail.com on 25 Dec 2011 at 8:42

Attachments:

• retry.zip

[GoogleCodeExporter]

This is clearly a nontrivial problem.  I am curious if this is the best way to 
solve it...

Original comment by wasserman.louis on 25 Dec 2011 at 12:42

[GoogleCodeExporter]

We are currently using this for years:
https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/pr
edicates/RetryablePredicate.java
https://github.com/jclouds/jclouds/blob/master/core/src/test/java/org/jclouds/pr
edicates/RetryablePredicateTest.java

It has limitations including no support for non-trivial shapes (ex. wait 
initially 10 seconds, then increase period over time), and sometimes you want 
to receive both the boolean result as well the last result tested.  The latter 
functionality is sketched here:

https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/pr
edicates/Retryables.java
https://github.com/jclouds/jclouds/blob/master/core/src/test/java/org/jclouds/pr
edicates/RetryablesTest.java

Clearly, we'd prefer something like this in guava as opposed to our codebase, 
so really excited about progress we can make here.

Original comment by adrian.f...@gmail.com on 4 Jan 2012 at 10:09

[GoogleCodeExporter]

personally, I like the customizability of comment 11

Original comment by adrian.f...@gmail.com on 4 Jan 2012 at 10:11

[GoogleCodeExporter]

It might be nice if the "Retryer" were a RetryingExecutorService and the 
response a ListenableFuture

Original comment by raymond....@gmail.com on 18 Jan 2012 at 9:09

[GoogleCodeExporter]

FYI, internally, we have:

- RetryingCallable (similar to the original suggestion and comment 11)
- RetryingFuture (repeated calls to a Supplier<Future>)
- Retry (bytecode magic to turn an object into a retrying proxy; unlikely to be 
open-sourced due to its dependencies)
- RetryingExecutor (still under review; similar to comment 15)

There is clearly demand, but there are a lot of possible approaches and a lot 
of work left to do, so I recommend using one of the existing solutions in the 
meantime.

Original comment by cpov...@google.com on 18 Jan 2012 at 9:19

• Changed title: Retry tool

[GoogleCodeExporter]

Here's an addition to add the list, resulting from jclouds proliferation of 
LoadingCache :)

       // deal with eventual consistency delay between bucket resources and their acls
       CacheLoader<String, AccessControlList> loader = RetryingCacheLoaderDecorator.newDecorator()
            .on(ResourceNotFoundException.class).exponentiallyBackoff()
            .decorate(
                new CacheLoader<String, AccessControlList>() {
                   @Override
                   public AccessControlList load(String bucketName) {
                      return client.getBucketACL(bucketName);
                   }

                   @Override
                   public String toString() {
                      return "getBucketAcl()";
                   }
                });

Original comment by adrian.f...@gmail.com on 15 May 2012 at 6:33

[GoogleCodeExporter]

+1 for comment 11 and 17's approach. I'm not sure I like the particulars of the 
builder pattern though, I'd like to save/serialize the builder so I could use 
it later.

.withStopStrategy() and .withWaitStrategy() could probably have variants 
directly in the builder, although having the ability to wire in your own 
strategies is very useful.

Original comment by emily@soldal.org on 15 May 2012 at 7:02

[GoogleCodeExporter]

Original comment by kevinb@google.com on 30 May 2012 at 7:43

• Added labels: Type-Addition
• Removed labels: Type-Enhancement

[GoogleCodeExporter]

This would be useful for me too, but so as not to make this a "me, too" post, 
here are a few links to others who have implemented this sort of thing, which 
can be used reference for a Guava version (which is what I'd prefer):

https://github.com/rholder/guava-retrying - not Guava but built on top of 
Guava; hence the name

http://grepcode.com/file/repo1.maven.org/maven2/org.springframework.batch/spring
-batch-infrastructure/1.0.0.FINAL/org/springframework/batch/retry/ - one from 
the Spring framework

Original comment by schni...@gmail.com on 4 Feb 2013 at 5:15

[GoogleCodeExporter]

There's also a .Net API called the transient fault handling application block 
which does the same thing:
http://msdn.microsoft.com/en-us/library/hh680905(v=pandp.50).aspx

Original comment by toellr...@gmail.com on 3 May 2013 at 6:15

[GoogleCodeExporter]

I have implemented both approaches:

1) simple: 
http://code.google.com/p/spf4j/source/browse/trunk/src/main/java/org/spf4j/base/
Callables.java

this implementation allows you to retry a callable based on Exception or 
returned result, and execute a callable before retry.

    public static <T> T executeWithRetry(final Callable<T> what, final Callable<Boolean> doBeforeRetry,
            final Predicate<? super T> retryOnReturnVal, final Predicate<Exception> retryOnException)
            throws InterruptedException 

2) 
sophisticated:http://code.google.com/p/spf4j/source/browse/trunk/src/main/java/o
rg/spf4j/concurrent/RetryExecutor.java

this is a executor based implementation that allows you to use your threads 
more efficiently.

Original comment by zolyfar...@yahoo.com on 11 May 2013 at 12:43

[GoogleCodeExporter]

I have tried an implementation too, trying to keep it simple:

https://github.com/jloisel/retrying-callable

I like the approach in #11 but i felt the need to separate concerns between 
retry on exception and retry on returned result.

Original comment by loisel.j...@gmail.com on 5 Jun 2013 at 7:14

[GoogleCodeExporter]

This issue has been migrated to GitHub.

It can be found at https://github.com/google/guava/issues/<id>

Original comment by cgdecker@google.com on 1 Nov 2014 at 4:15

• Added labels: MigratedToGitHub

[GoogleCodeExporter]

Original comment by cgdecker@google.com on 1 Nov 2014 at 4:18

• Changed state: Migrated

[GoogleCodeExporter]

Original comment by cgdecker@google.com on 3 Nov 2014 at 9:09

• Added labels: Restrict-AddIssueComment-Commit