exponential backoff?

[vsoch]

Does cloudstorage use exponential backoff (typically done via retrying)? It could be that some of the individual (underlying) backends implement this (I remember it suggested in a Google tutorial a while back but I needed to wrap the functions myself), but it would be good to have it as an option if this isn't the case, or otherwise documented. Thanks!

[scottwernervt]

Does cloudstorage use exponential backoff (typically done via retrying)?

cloudstorage does not implement exponential backoff. However, it is just a wrapper around common storage backend libraries like boto3 which does have a retry mechanism built in.

You could definitely wrap the functions yourself using tenacity (retrying looks abandoned), e.g.:

from tenacity import (
    retry,
    retry_if_exception_type,
    stop_after_attempt,
    wait_fixed,
)
from cloudstorage.drivers.rackspace import CloudFilesDriver
from cloudstorage.exceptions import NotFoundError

storage = CloudFilesDriver('key', 'secret')

@retry(retry=retry_if_exception_type(NotFoundError),
       wait=wait_fixed(2),
       stop=stop_after_attempt(2))
def wait_for_container_to_exist():
    print('Getting container...')
    return storage.get_container('cloud-storage-test-1519657273-xbiqomad')

wait_for_container_to_exist()
# Getting container...
# Getting container...

[vsoch]

fantastic, thank you! Tenacity looks like it works in almost the same way, and this is a really good example to have. Thanks for the speedy response, closing issue!