aws-deadline / deadline-cloud-test-fixtures

This package contains pytest fixtures that are used to test AWS Deadline Cloud Python packages
Apache License 2.0
9 stars 13 forks source link

fix: increase ec2 instance_status_ok waiter timeout thresholds #47

Closed jusiskin closed 9 months ago

jusiskin commented 9 months ago

What was the problem/requirement? (What/Why)

The EC2InstanceWorker class can sometimes error on a retry/timeout when waiting for the EC2 instance status to become ok.

botocore.exceptions.WaiterError: Waiter InstanceStatusOk failed: Max attempts exceeded

What was the solution? (How)

Increase the thresholds from their defaults:

retry configurable default new value
sleep_amount 15 15
max_attempts 40 60

This effectively goes increases the total time for the EC2 instance to become ready from 10 minutes → 15 minutes.

What is the impact of this change?

There is a lower probability of the tests failing due to the EC2 instance not being ready in within the threshold allowed by the boto3 waiter.

How was this change tested?

Ran the worker agent tests using these code changes

Was this change documented?

No

Is this a breaking change?

No