Cadvisor ignores container renaming

[GregoryKutuzov]

When a container is renamed in Docker, cAdvisor does not update the container name, but continues to provide the data under the old container name. This means that our storage driver stores irrelevant data and does not reflect the changes that took place on the system.

[ghost]

+ I have the same problem. After container rename, cAdvisor just lost it and only option is to restart cAdvisor.

[vishh]

cAdvisor doesn't reconcile the state of a container after it discovers it in the beginning. We will have to periodically inspect the container and update the internal state. Does anyone have some spare cycles to fix this?

On Thu, Apr 7, 2016 at 5:11 AM, psychoche notifications@github.com wrote:

+ I have the same problem. After rename cAdvisor just lost container and only option is restart cAdvisor.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/google/cadvisor/issues/977#issuecomment-206842730

[Cellane]

Hello, just found this ticket while I was writing my own ticket about the same problem. I was wondering if there’s any plan or progress on this issue. I’m afraid I might not possess the skills necessary to fix this myself, but if it helps, I can present a scenario in which the existence of this issue breaks any usefulness of cAdvisor.

For my tiny personal server, I use Dokku for managing and deploying applications. Dokku’s default behaviour when deploying a container is to let Docker generate a random name for it at first, and once the container starts and passes health checks, it is renamed to a more useful name in the form of <app-name>.<proc-type>.<#>, for example, cadvisor.web.1 (and the container that is being replaced is renamed to <app-name>.<proc-type>.<#>.<unix-timestamp> until it is despawned a minute later).

What that means is that if all of my apps are started before cAdvisor (or rather, renamed to correct names before cAdvisor starts), everything is fine. But as soon as I start pushing new versions of apps, or changing their configuration (by replacing old containers), the level of useful information I’m getting out of cAdvisor gets progressively worse, as all new apps are forever reporting under the randomly generated name, which is unfortunately not useful at all.

I’m not sure how common my use case/scenario is, but I wonder if there’s any plan to fix this in the future. For the record, I’m observing the issue with cAdvisor 0.34.0.

[badsyntax]

@Cellane I'm in a similar situation. I was wondering if this workaround is sufficient:

Assuming you're using prometheus, we can graph prom queries in grafana by docker labels instead of container name. Eg for memory usage:

sum(container_memory_rss{name=~".+"}) by (container_label_com_dokku_app_name, container_label_com_dokku_dyno)

Legend:

{{container_label_com_dokku_app_name}}.{{container_label_com_dokku_dyno}}

Seems to me this should be sufficient as the labels won't change, but I'm not that expereinced with this stuff and don't have confidence this is a good workaround.