Open ryan-beisner opened 7 years ago
Hey @ryan-beisner--
It looks like you're just using the default timeout for a charmstore object--you can pass in a timeout value when you create one--see https://github.com/juju/theblues/blob/develop/theblues/charmstore.py#L24.
Would that fix things for you?
Unfortunately, I don't have control over that lever, as theblues is instantiated by Amulet which is wrapped by Bundletester. Ref: traceback.
What's a reasonable timeout if not 3 seconds?
On Thu, Nov 10, 2016 at 2:13 PM Ryan Beisner notifications@github.com wrote:
Unfortunately, I don't have control over that lever, as theblues is instantiated by Amulet which is wrapped by Bundletester. Ref: traceback.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/juju/theblues/issues/49#issuecomment-259779744, or mute the thread https://github.com/notifications/unsubscribe-auth/AAET1evyolsS8xs2fJZmgyZB93V2ZYJ-ks5q82yugaJpZM4Kt6ej .
@marcoceppi That depends on the Interwebs weather on any particular day, which makes this tricky.
Typically, to handle these sort of things, you'd want to have a low initial timeout (3s is probably ok), wrapped in a retry loop with a backoff (perhaps 3s, 6s, 12s, 24s) and a max_wait (1m). If it doesn't come through in 1m, it's probably never going to, and it should bail.
We have been seeing sustained and noticeable charm store api timeouts in OpenStack Charm CI. This causes false test failures in our charm dev/test gate.
Can this be made more resilient and/or configurable, such as retry threshold and timeout backoffs?