aws-samples / serverless-jenkins-on-aws-fargate

MIT No Attribution
123 stars 111 forks source link

ECS Service Deployment Downtime #38

Open jgard opened 1 month ago

jgard commented 1 month ago

The ECS Service definition committed by @krithivasan contains:

  // Assuming we cannot have more than one instance at a time. Ever. 
  deployment_maximum_percent         = 100
  deployment_minimum_healthy_percent = 0

Which results in downtime as the old task is stopped before the new task comes up. I've tested with min=100/max=200 and it worked just fine. No downtime for the UI, and a script invoking jobs constantly via the REST API encountered no issues launching the jobs, and they all ran fine.

Potentially relevant details:

Wondering if anyone can speak authoritatively to the risk in letting controller container tasks run concurrently for a few minutes. I'd appreciate a technical explanation (like, "both processes will touch this file and could cause a deadlock"). But I'd take any real world anecdotes or documented best practices.

I'm also going to chime in similarly at https://community.jenkins.io/t/error-more-than-one-instance-sharing-var-lib-jenkins/8599

Thanks!