When the S3 service experiences service-issues, it can cause the auto-recovery enabling cron jobs to go out to lunch and pile up. Need to ensure that said cron-jobs don't run past their run widow (constrain to < 20 minutes or whatever RPO - 2m works out to).
Fix recommendation:
Encapsulate auto-recovery enabling cron-job in a timeout() block
Problem Description:
When the S3 service experiences service-issues, it can cause the auto-recovery enabling cron jobs to go out to lunch and pile up. Need to ensure that said cron-jobs don't run past their run widow (constrain to < 20 minutes or whatever
RPO - 2mworks out to).Fix recommendation:
Encapsulate auto-recovery enabling cron-job in a
timeout()block