search

kopeio / etcd-manager

operator for etcd: moved to https://github.com/kubernetes-sigs/etcdadm
Apache License 2.0
164 stars 45 forks source link

How to get metrics from etcd after migration to etcdmanager? #286

Closed michalschott closed 3 years ago

michalschott commented 4 years ago

Hi,

I'm using datadog for monitoring solution, but I feel like this is more generic question. Previously (before etcdmanager was introduced) port 4001 and 4002 were accessible, and if I recall correctly these are not exposed on nodes anymore.

Previously, our datadog check was configured like this:

---
init_config:
instances:
  - url: http://etcd-a.internal.CLUSTER_NAME:4001

What's the correct url to use now?

carpenterbees commented 4 years ago

After upgrade to etcd v3, metrics endpoint is available only on https port which requires auth with TLS certificate. Your monitoring client has to provide certificate to be able to read from etcd nodes.

Another solution is to configure custom url: Support listen-metrics-url

Problem with this solution - how to pass argument or environment variable via kops cluster manifest.

michalschott commented 4 years ago

Hi @carpenterbees, thanks for pointing me to proper PR. I was able to expose the endpoint and scratch the metrics again. Raised a PR in kops to see if we can enable this by default.

themoah commented 4 years ago

@michalschott can you, please, share how you've exposed metrics ? I'm having same problem with datadog. thanks !

michalschott commented 4 years ago

@themoah https://github.com/kubernetes/kops/pull/8162 Until this is merged, I'm using dummy-etcd pods approach suggested by DD support.

themoah commented 4 years ago

@michalschott can you share the configuration of the pods ? Appreciate the help