Add option to retry processor at exponentially increasing interval

[jiexinhuang]

Context

ESP process can retry processing an event in case of errors, this is helpful to handle random errors on unreliable dependencies like third party APIs, but retry every second could flood our error trackers like Rollbar and generate unnecessary cost.

Changes

• Add 3 built-in error handlers: ConstantRetry, NoRetry and ExponentialBackoffRetry
• Add a config to EventSourcery to config ErrorHandlerClass, a client app could use built-in error handlers or use a custom one.
• Remove stop_on_failure option for ESPProcess, this option is superseded by ErrorHandler
• Update README

Consideration

This is a non-breaking change and should not cause issues for client apps when upgrading.

[orien]

Algorithm/behaviour looks good. I think now's a good time to refactor. How might it look with a class dedicated to this concern?

[mjward]

Sorry, bit late to the party. I think having this behaviour exist in the framework is awesome. It provides the missing balance between retry forever and die on failure, which is a nice fallback for transient errors.

To throw some further context at the PR, are you able to provide some reasoning behind why you felt you needed this for RSS? (maybe an example scenario)

[jiexinhuang]

@mjward One scenario I can think of is when Recurly is down, the reactor that calls Recurly API to create and update subscriptions will throw exceptions and we do want to retry in this case, but retry every second could potentially cost a lot of our Rollbar quota.

[jiexinhuang]

@orien @stevehodgkiss would you mind having another look at these refactor commits?

[jiexinhuang]

@orien @stevehodgkiss Could you guys have another look please 😄