litl / backoff

Python library providing function decorators for configurable backoff and retry
MIT License
2.61k stars 148 forks source link

Feature Request: support direct invocation with functools.partial #210

Open mikepii opened 12 months ago

mikepii commented 12 months ago

I would like to be be able to use the backoff library to decorate a Callable created using functools.partial so that I can:

Example of proposed backoff usage with functools.partial:

from functools import partial
from typing import Any, Callable

import backoff

def main() -> None:
    run_http_client_operation_with_retries(partial(foobar, baz=123))

def run_http_client_operation_with_retries(operation: Callable[[], Any], max_time: Float) -> Any:
    create_callable_with_retries = backoff.on_exception(
        lambda: backoff.constant(20), ApiRetryableError, max_time=max_time
    )
    operation_with_retries = create_callable_with_retries(operation)
    return operation_with_retries()

def foobar(baz):
    ...

class ApiRetryableError(Exception):
    pass

Currently, this is close to working, but it fails to generate the "backing off" log message (with backoff 2.2.1 and Python 3.10.6):

  File "/.../lib/python3.10/site-packages/backoff/_common.py", line 97, in _log_backoff
    log_args = [details['target'].__name__, details['wait']]
AttributeError: 'functools.partial' object has no attribute '__name__'

A Python lambda can be used as a workaround (run_http_client_operation_with_retries(lambda: foobar(baz=123))), but then the log message is unclear (Backing off <lambda>(...) ...). One advantage of functools.partial is that it provides a clearer function name when you do repr(functools.partial(...)).

Using backoff as a context manager was proposed in #198 and #18. That would solve the same problem of being able to specify backoff parameters dynamically. However, it's impossible to implement a context manager that re-runs the code block it manages (see https://github.com/litl/backoff/issues/18#issuecomment-262261996).