Flakey race condition in backoff

[asottile]

test.plugins.sql_reporter_test RetryTestCase.test_can_fail ... SQL connection failed, retrying in 0.01 seconds (giving up in 0.44 seconds)...
SQL connection failed, retrying in 0.01 seconds (giving up in 0.44 seconds)...
SQL connection failed, retrying in 0.02 seconds (giving up in 0.43 seconds)...
SQL connection failed, retrying in 0.02 seconds (giving up in 0.43 seconds)...
SQL connection failed, retrying in 0.03 seconds (giving up in 0.41 seconds)...
SQL connection failed, retrying in 0.03 seconds (giving up in 0.41 seconds)...
SQL connection failed, retrying in 0.04 seconds (giving up in 0.38 seconds)...
SQL connection failed, retrying in 0.04 seconds (giving up in 0.38 seconds)...
SQL connection failed, retrying in 0.05 seconds (giving up in 0.34 seconds)...
SQL connection failed, retrying in 0.05 seconds (giving up in 0.34 seconds)...
fail: test.plugins.sql_reporter_test RetryTestCase.test_can_fail
Traceback (most recent call last):
File "./test/plugins/sql_reporter_test.py", line 272, in test_can_fail
self.reporter._connect()
File "/opt/python/2.6.9/lib/python2.6/contextlib.py", line 23, in __exit__
self.gen.next()
AssertionError: No exception was raised (expected <class 'sqlalchemy.exc.OperationalError'>)

FAIL in 0.22s

[bukzor]

Two strange things to note: the retry wait time isn't incrementing as it normally should. Secondly, the time limit also isn't decreasing as it should. In fact the wait/limit numbers seem to come in matched pairs. It feels like two threads are running this function.

[bukzor]

Fixed by #267