search

DataDog / datadog-agent

Main repository for Datadog Agent
https://docs.datadoghq.com/
Apache License 2.0
2.84k stars 1.19k forks source link

Proxy appears to preclude autodiscovery on Kubernetes #4618

Open DerekV opened 4 years ago

DerekV commented 4 years ago

I was learning about autodiscovery for our node agents on Kubernetes. The node agent does detect a local pod that it may monitor (in the observed case, nginx), but the check fails because we have set DD_PROXY_HTTP. Since it is using the pod IP and not a hostname, and because DD_PROXY_NO_PROXY seems to be simply doing a string equality comparison , I can not exclude pod IPs.

Initial thoughts on fixing this - Could allow IP prefixes, CIDRs and hostname suffixes in DD_PROXY_NO_PROXY or proxy.noProxy config, and/or we could detect the kubernetes environment and automatically exclude the pod IP range, and/or do not use the proxy for autodiscovered checks in any case.

The latter could maybe be accomplished by having a check config flag "use_proxy" or "ignore_proxy", and autodiscovery can set this to true, and the check can pass this to the http client with the request.

DylanLovesCoffee commented 4 years ago

@DerekV :wave: edit: I had spoken too soon. Perhaps you could add per check proxy configurations into your pod annotations/configMap?

DerekV commented 4 years ago

I have had a support thread going for a while ( 290503 ) . Setting skip_proxy: true for nginx in a configMap makes no difference. I have not confirmed if this is isolated to nginx or if it effects all http based autodiscovery checks.