search

hashicorp / waypoint-helm

Helm chart to install Waypoint and other associated components.
Mozilla Public License 2.0
29 stars 14 forks source link

Auto clean up ODR Task containers #38

Closed philthynz closed 2 years ago

philthynz commented 2 years ago

Issue On demand runner task containers seem to stay in "Succeed" status and do not auto clean. There ends up being 100's of containers in the Waypoint namespace which makes it hard to filter through tasks to debug.

Solution Can there be a TTL on these containers? As mentioned in: https://kubernetes.io/docs/concepts/workloads/controllers/job/#clean-up-finished-jobs-automatically

Alternative If not possible we could use a cron that runs kubectl delete pod --field-selector=status.phase==Succeeded -n waypoint.

Additional context After a deployment using an ODR Task the container’s last state is “Succeeded”, and there ends up being hundreds of these lingering around. Each deployment task spins up a new container called waypoint-task-<ID>.

philthynz commented 2 years ago

Seems to be fixed in 0.8.2, and after I patched the pods that were left in a "Terminating" state. https://discuss.hashicorp.com/t/kubernetes-odr-task-containers-waypoint-task-id-clean-up/40081/3