awslabs / ecs-refarch-service-discovery

An EC2 Container Service Reference Architecture for providing Service Discovery to containers using CloudWatch Events, Lambda and Route 53 private hosted zones.
Apache License 2.0
445 stars 81 forks source link

Only two ECS instances out of four are working #3

Closed lucacome closed 8 years ago

lucacome commented 8 years ago

Every time I launch the CloudFormation Stack, no matter the number of ECS instances I choose, only 2 are appearing in ECS. If I choose 4 instances I can see that 4 EC2 instances are created, but only 2 available in ECS.

Logging in the other two shows that there is a problem with docker:

$ cat /var/log/docker \nTue Jun 7 16:30:39 UTC 2016\n time="2016-06-07T16:30:39.219356348Z" level=info msg="API listen on /var/run/docker.sock" \nTue Jun 7 16:31:12 UTC 2016\n time="2016-06-07T16:31:12.241744210Z" level=info msg="New containerd process, pid: 2772\n" time="2016-06-07T16:31:14.023350929Z" level=fatal msg="Error starting daemon: error initializing graphdriver: Device is Busy"

schmutze commented 8 years ago

Hi @lucacome,

I was able to replicate the issue you saw and agree updating the AMI should resolve the issue. I've done so in: https://github.com/awslabs/ecs-refarch-service-discovery/commit/02bf7f35ba6d979813789cf322d55cb36a47e93f

Please let me know if you continue to see the issue.