Provide better visibility into containers that cannot be killed

cloudfoundry / guardian

containers4life

Apache License 2.0

77 stars 44 forks source link

Provide better visibility into containers that cannot be killed #429

Closed geofffranks closed 9 months ago

geofffranks commented 9 months ago

When container processes get stuck blocked in kernel code (e.g. a connection disappears mid-write to an NFS volume store), the processes are unkillable even with kill -9.

This change adds the following:

updated log messages when kill -9 has failed, indicating that the VM running garden must be rebooted.
a new UnkillableContainers metric is emitted that can be used to determine if a VM has any containers that garden was unable to kill.

cf-gitbot commented 9 months ago

We have created an issue in Pivotal Tracker to manage this. Unfortunately, the Pivotal Tracker project is private so you may be unable to view the contents of the story.

The labels on this github issue will be updated when the story is started.