Healthcheck on 44554 doesn't detect hung/broken dockerd - Githubissues

docker-archive / for-aws

92 stars 26 forks source link

Healthcheck on 44554 doesn't detect hung/broken dockerd #192

Open molexx opened 5 years ago

molexx commented 5 years ago

Expected behavior

When dockerd hangs or is otherwise broken, the healthcheck service running on 44554 should report it is down so that AWS can restart the machine.

Actual behavior

The service on 44554 returns 'LGTM' despite:

portainer claims the swarm is down - 'endpoint is unreachable'
port 22 is not connectable - suggesting that the shell-aws container is down
port 2375 connects but times out
port 2376 is not connectable

Information

docker-diagnose on a worker produces no output. docker info on a worker looks ok, it still reports the manager's IP address

Investigation is restricted due to not being able to ssh in to the manager machine.