Garbage collect unused docker images on edge node

[cgiroua]

Currently when a service version is updated, the old container is removed but the image remains. Over time, this could fill up the edge-node disk.

We should look into ways of handling this (max image count per service, or other mechanisms).

[mustafamg]

Is there any update regarding this issue or a recommended approach to handle removing old images?

[johnwalicki]

This problem is real. I have active edge nodes which deploy updated containers. These AI containers are large and often fill up the SDcard storage with old revisions. Garbage cleanup would be useful.

[johnwalicki]

The trouble with garbage clean up is the agent doesn't really know which old container images previously were service containers it managed. The edge node might have dozens of containers. Some might have been built locally by the developer, some via manual docker pull, some might have been deployed to the edge node via a Horizon service definition/pattern/policy and agreement.

Once anax terminates an agreement and stops the service container, it does not store a history of unused containers. If the anax agent just blindly lists unused images and does a docker image rm 0123456789abcdef it might delete container images that are unrelated to its previous agreements.

To implement this, the agent would either :

1. Store a list of old agreements / service container images and dates these agreements were formed in the anax database. Filter on some configurable date (delete images older than 7 days or before some timestamp)
2. Opt in to delete all unused container images older than some timestamp (potentially dangerous)