VictoriaMetrics / operator

Kubernetes operator for Victoria Metrics
Apache License 2.0
410 stars 141 forks source link

bug: two different vmagent's in different namespaces can get the same credentials.. #891

Open ThomasADavis opened 4 months ago

ThomasADavis commented 4 months ago

If you:

a) create an vmagent, in namespace a, with a name of 'vmagent', then b) create a vmnodescrape for cadvisor and verify it works. c) create a second vmagent, in namespace b, with a name of 'vmagent', then.. d) watch the vmnodscrape for cadvisor fail after the second vmagent is created. This may take a minute or two to show up. e) delete the 2nd vmagent, wait for and watch the 1st vmagent cadvisor scrapes start to work again.

when you check the clusterroles/clusterrolbindings, you'll find that the second vmagent has corrupted the first vmagent's credentials.

It will change back and forth each time the operator updates the vmagents.

Workaround is to give every vmagent running on the system a unique name and not to rely on namespacing.

The issue is clusteroles/clusterbindings are not namespaced, and must be unique for each vmagent, and they are not when you do the above.