aws / amazon-mwaa-docker-images

Apache License 2.0
24 stars 11 forks source link

Letting each condition control its frequency #150

Open ashishgo-aws opened 7 hours ago

ashishgo-aws commented 7 hours ago

Using throttling, each condition was being checked once per minute. But a condition may need to perform its check at a different pace like for example the TaskMonitor condition. This change applies throttling at individual condition instead of applying it generically at the subprocess level. The TaskMonitor condition will not be executed once per 10 seconds instead of one minute. This will allow for faster evaluation of worker idleness which would help in scaling down the worker fleet a lot more effectively.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.