search

deis / charts

(OBSOLETE) Helm Classic v1 Charts for Deis Workflow
https://deis.com/workflow/
MIT License
45 stars 36 forks source link

feat(workflow-dev-e2e): add spotify/docker-gc to e2e #352

Closed bacongobbler closed 7 years ago

bacongobbler commented 7 years ago

As seen in our GKE clusters, the docker images for the applications spawned by e2e are never reaped. This causes thousands of images to persist, all identical to each other on a single node and can trigger etcd issues.

See https://github.com/deis/controller/issues/1082#issuecomment-248079239 for some background context.

bacongobbler commented 7 years ago

@mboersma mentioned to me during manual testing that this is a one-off container. That is, it runs to completion then dies. It looks like the container is unhealthy in a CrashLoopBackoff but in reality it's just running a few times during the e2e job and that's all we really need.

Open to other suggestions to improve upon this, but I think this is good enough to ship so we stop breaking GKE :)

mboersma commented 7 years ago

I made a trivial fork of spotify/docker-gc that keeps the container from exiting by running the script every two hours via crond. You can test it using quay.io/mboersma/spotify-gc:0.1.2 if you're curious. Maybe something like that could be included with Deis Workflow itself, although it might be better to start from deis/base.