search

prometheus-community / elasticsearch_exporter

Elasticsearch stats exporter for Prometheus
Apache License 2.0
1.93k stars 791 forks source link

log can still contain clear text credential #299

Open phsiao opened 5 years ago

phsiao commented 5 years ago

Looks like there are still some paths exist that do not handle redaction as expected.

Using a sandbox cluster where all user credentials are randomly generated, before the cluster is ready the prometheus-exporter would show this:

level=info ts=2019-10-24T18:02:45.076664968Z caller=clusterinfo.go:200 msg="triggering initial cluster info call"
level=info ts=2019-10-24T18:02:45.07681672Z caller=clusterinfo.go:169 msg="providing consumers with updated cluster info label"
level=error ts=2019-10-24T18:02:50.077032958Z caller=clusterinfo.go:253 msg="failed to get cluster info" err="Get https://admin:t5%2AzdjfO3e%3C3&c88p%5DUO1M04%2FvNl%3E%7Dp$@elasticsearch:9200/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
level=error ts=2019-10-24T18:02:50.173526771Z caller=clusterinfo.go:174 msg="failed to retrieve cluster info from ES" err="Get https://admin:t5%2AzdjfO3e%3C3&c88p%5DUO1M04%2FvNl%3E%7Dp$@elasticsearch:9200/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
level=info ts=2019-10-24T18:02:55.076968842Z caller=main.go:153 msg="initial cluster info call timed out"
level=info ts=2019-10-24T18:02:55.274039775Z caller=main.go:188 msg="starting elasticsearch_exporter" addr=:9114

This is our kubernetes pod container definition

  containers:
  - args:
    - --es.uri=https://$(PROMETHEUS_EXPORTER_USER):$(PROMETHEUS_EXPORTER_PASS)@elasticsearch:9200
    - --es.all
    - --es.ca=/usr/share/elasticsearch/config/elk-root-ca.pem
    - --es.client-private-key=/usr/share/elasticsearch/config/elk-key.pem
    - --es.client-cert=/usr/share/elasticsearch/config/elk-crt.pem
    - --es.ssl-skip-verify
    - --web.listen-address=:9114

where the two env vars are injected from secrets.

zwopir commented 5 years ago

Thanks for reporting!

it's a bit strange, as the leaked password is returned by a http.Client.Do() call error, which should be correctly redacted. See https://github.com/golang/go/blob/master/src/net/http/client.go#L594

I have a closer look what's wrong here

zwopir commented 5 years ago

this is fixed in go1.13, but we're still on go1.12. See https://github.com/golang/go/issues/31808. I'll try to switch to go1.13 as fast as possible.