Implement exponential backoff retry strategy

[yawaramin]

• Motivation
  • To respect download retry guidelines provided by Google etc. https://developers.google.com/api-client-library/java/google-http-java-client/backoff?hl=en
• Input
  • The com.mediative.eigenflow.domain.Retry type which implements potential retry strategies
• Output
  • The above type with the option to specify an exponential backoff
• Test
  • A sample (dummy) job implementing some trivial failing action and demonstrating that an exponential backoff strategy was followed.

[yawaramin]

One possible way to do this is to redefine ExecutionPlan#retryStrategy to be a stream of retry descriptions (Retry objects); we retry as long as there are more elements in the stream, obeying the described interval. Then the exponential backoff strategy could be something like

object Retry {
  /**
  Returns a stream of retry descriptions following the exponential backoff strategy.
  */
  def exponentialBackoff(initialInterval: FiniteDuration, increaseFactor: Int, attempts: Int) =
    Stream.iterate(Retry(initialInterval), attempts) { retry =>
      retry.copy(interval = increaseFactor * retry.interval)
    }

  val noRetry = Stream.empty
}

[yawaramin]

Of course, with this re-definition of ExecutionPlan#retryStrategy, the implication is that Retry#attempts is no longer needed and a retry description becomes semantically just an interval to wait: case class Retry(interval: FiniteDuration) extends AnyVal. The number of attempts is captured by the length of the retry strategy stream.