Closed flaky-bot[bot] closed 1 month ago
Looks like this issue is flaky. :worried:
I'm going to leave this open and stop commenting.
A human should fix and close this.
When run at the same commit (bffae0451f0d6dd4e1346cb513db20e0682c507b), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).
The test cloud-devrel/python-docs-samples/python3.8/periodic
has been failing since Jan, 2024 ( we don't have history of runs before that ) consistently.
While it looks like there are multiple kinds of failures. The most consistent ( and not flaky ) that I notice are these:
test_create_monitor_ca_policy
: StatusCode.RESOURCE_EXHAUSTED
403 permission issue
@cojenco @andrewsg
@bajajneha27 I took a look and it seems like the periodic runs tests for all packages (cloud products) in this repo. There are 3 storage transfer tests we own that are erroring out during pytest fixture aws bucket creation setup: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/2299beb49e5639d5ea46e6884b2963f6019460f3/storagetransfer/conftest.py#L230
The error is TooManyBuckets
, and based on aws documentation, it seems like we've hit the 100 bucket quota per aws account.
botocore.exceptions.ClientError: An error occurred (TooManyBuckets) when calling the CreateBucket operation: You have attempted to create more buckets than allowed
It could be that some buckets were not successfully deleted? I also don't know how many tests are using the same aws account. @JesseLovelace @danielbankhead do you have more context on the aws account used here? Could you point us to where/how the aws account is setup?
File "/workspace/storagetransfer/conftest.py", line 230, in aws_source_bucket
s3_client.create_bucket(Bucket=bucket_name)
File "/workspace/storagetransfer/.nox/py-3-8/lib/python3.8/site-packages/botocore/client.py", line 565, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/workspace/storagetransfer/.nox/py-3-8/lib/python3.8/site-packages/botocore/client.py", line 1021, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (TooManyBuckets) when calling the CreateBucket operation: You have attempted to create more buckets than allowed
-------- generated xml file: /workspace/storagetransfer/sponge_log.xml ---------
=========================== short test summary info ============================
ERROR aws_request_test.py::test_aws_request - botocore.exceptions.ClientError...
ERROR aws_request_test.py::test_aws_request_apiary - botocore.exceptions.Clie...
ERROR event_driven_aws_transfer_test.py::test_event_driven_aws_transfer - bot...
=================== 17 passed, 3 errors in 92.39s (0:01:32) ====================
Note: #11067 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.
commit: bffae0451f0d6dd4e1346cb513db20e0682c507b buildURL: Build Status, Sponge status: failed