Open scottwernervt opened 6 years ago
This just a travis cli / testing issue, right? It shouldn't effect end users?
Yes, it is only a testing issue and does NOT effect end users.
You can try make following improvements and it will useful not only for tests
--- a/src/cloudstorage/drivers/google.py
+++ b/src/cloudstorage/drivers/google.py
@@ -8,6 +8,9 @@ from datetime import datetime, timedelta
from http import HTTPStatus
from typing import Any, Dict, Iterable, List # noqa: F401
+from urllib3.util.retry import Retry
+from requests.adapters import HTTPAdapter
+
# noinspection PyPackageRequirements
from google.auth.exceptions import GoogleAuthError
@@ -110,6 +113,13 @@ class GoogleStorageDriver(Driver):
self._client = storage.Client()
+ retries_strategy = Retry(
+ total=5,
+ backoff_factor=0.1,
+ status_forcelist=[408, 429])
+ http = self._client._http
+ http.mount('https://', HTTPAdapter(max_retries=retries_strategy))
+
def __iter__(self) -> Iterable[Container]:
for bucket in self.client.list_buckets():
yield self._make_container(bucket)
Google driver tests still randomly fail due to API rate limiting even after introducing delays. Need to rework how we use fixtures for containers and blobs.
Ideas: