search

rholder / retrying

Retrying is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything.
Apache License 2.0
1.91k stars 157 forks source link

Support for marked wait #51

Closed bodenr closed 8 years ago

bodenr commented 8 years ago

Often times retrying is done in a stepping manner whereupon the first N attempts wait X ms, the next M attempts wait for Y ms, etc. This patch implements support for such waiting by introducing the notion of 'wait markers'. Wait markers are simply tuples that define a set of attempts and their corresponding waits to go into effect.

A handful of unit tests are also included.

bodenr commented 8 years ago

@jd @harlowja @rholder I realize this PR has conflicts now, but I'd like to understand your initial impressions of this code to determine if it's worth my time continuing here. Thanks

jd commented 8 years ago

This looks simple enough to be done with a custom wait_func so I wonder if it's worth it?

bodenr commented 8 years ago

@jd well almost anything in this respect could be done with a custom wait func couldn't it? : )

The idea was to enhance the base functionality of retrying by introducing this flexible mechanism. IMHO it could be pretty useful.

jd commented 8 years ago

You're right. I think we should just stop adding more kwargs to init, and just use partial to provide pre-defined wait_funcs in this regard.

bodenr commented 8 years ago

@jd sorry I'm not sure I completely follow your last comment.

Are you inferring we should update how retrying supports waiting, or are you suggesting in cases like this (add new wait behavior) we house custom the logic in the consumer and just use a consumer's custom wait func?

jd commented 8 years ago

My suggestion is to do this

@retrying.retry(wait_func=retrying.wait_markers((2, 10), (4, 20)))
def foobar(): pass

and in retrying.py

def wait_markers(*args):
   @wraps
    def _wrap(f):
         sleep(using args)
   return _wrap

See what I mean? :)

bodenr commented 8 years ago

Yeah so what I referred to (perhaps poorly) as housing the logic in the consumer. I'm fine with that, but does your comment about not adding any more args to init infer that maybe retrying could use a little redesign/refactor :)

jd commented 8 years ago

We all know it needs some refactor. We're waiting for @rholder to merge PR and release (more often) :)

bodenr commented 8 years ago

I'll close this one out then; as we agreed for now the custom wait func won't live in retrying. If I'm incorrectly closing this, please feel free to reopen.

Thanks

jd commented 8 years ago

Don't get me wrong: I'd be ok to have it living in retrying, just not added in init kwargs.

My example above shows that :)