prometheus bearer token

[bradrydzewski]

I apologize if this is the wrong place to make such a request, but I'm having difficulty going through the support channels. I would like to scrape a prometheus /metrics endpoint with the datadog agent, however, the metrics endpoint is protected and requires a bearer token for authorization.

When you configure a scraper in prometheus you can specify a bearer_token parameter [1]. It would be great if this could be configured with the datadog agent as well (If this is already possible, my apologies). Thanks!

# Sets the `Authorization` header on every scrape request with
# the configured bearer token. It is mutually exclusive with `bearer_token_file`.
[ bearer_token: <secret> ]

[1] https://prometheus.io/docs/prometheus/latest/configuration/configuration/

[hkaj]

Thanks for reaching out, I have a PR open to document that actually: https://github.com/DataDog/integrations-core/pull/2585 So basically you need to expose the bearer token as an environment variable in the datadog agent pod, and then use add this to your app pod annotations:

         ad.datadoghq.com/slave.check_names: '["openmetrics"]'
         ad.datadoghq.com/slave.init_configs: '[{}]'
         ad.datadoghq.com/slave.instances: '[{"prometheus_url": "http://%%host%%:<MY_EXPORTER_PORT>/metrics","namespace": "<MY_APP>","metrics": ["*"],"extra_headers": {"Authorization": "Bearer %%env_<MY_TOKEN_ENV_VAR>%%"}}]'

And the agent will interpolate the %%env_TOKEN%% part with your token. Does that address your use case?

[grosser]

putting tokens into the env is not really safe, support for bearer_token_file would be great

[clamoriniere]

putting tokens into the env is not really safe, support for bearer_token_file would be great

Hi @grosser To retrieve securely a bearer token, you can now use the "secret backend" feature.

In your case the secret backend implementation can read the bearer token in a file. You can use the example script present in the agent docker image [/readsecret.py](https://docs.datadoghq.com/agent/guide/secrets-management/?tab=linux#helper-script-for-autodiscovery), it do exactly what you need.

[grosser]

that's awesome!

On Sun, May 24, 2020 at 10:18 AM Cedric Lamoriniere < notifications@github.com> wrote:

putting tokens into the env is not really safe, support for bearer_token_file would be great

Hi @grosser https://github.com/grosser To retrieve securely a bearer token, you can now use the "secret backend" feature https://docs.datadoghq.com/agent/guide/secrets-management/?tab=linux#using-secrets .

In your case the secret backend implementation can read the bearer token in a file. You can use the example script present in the agent docker image /readsecret.py, it do exactly what you need.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/DataDog/datadog-agent/issues/2667#issuecomment-633262596, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACYZY5HIK47FH6Q3X2I4LRTFJFFANCNFSM4GD2CCJA .

[pmalek]

Isn't that what bearer_token_auth and bearer_token_path do?

https://docs.datadoghq.com/integrations/guide/prometheus-host-collection/#parameters-available

[grosser]

yeah that looks right, can probably close this issue @bradrydzewski

[pmalek]

BTW any clue why https://docs.datadoghq.com/integrations/guide/prometheus-host-collection/#parameters-available mentions openmetrics_endpoint but https://github.com/DataDog/integrations-core/blob/14f280a0f339e53a7703c86d687a8ca32aa683ed/datadog_checks_base/datadog_checks/base/checks/openmetrics/base_check.py#L15 uses prometheus_url?

I've spent half a day on trying to figure out why my Kubernetes auth token isn't valid or is not sent and it seems that using openmetrics_endpoint will not send it, whereas prometheus_url will 🤷

[grosser]

maybe best to open a new issue to get more eyes

On Sat, Mar 9, 2024 at 12:29 AM Patryk Małek @.***> wrote:

BTW any clue why https://docs.datadoghq.com/integrations/guide/prometheus-host-collection/#parameters-available mentions openmetrics_endpoint but https://github.com/DataDog/integrations-core/blob/14f280a0f339e53a7703c86d687a8ca32aa683ed/datadog_checks_base/datadog_checks/base/checks/openmetrics/base_check.py#L15 uses prometheus_url?

I've spent half a day on trying to figure out why my Kubernetes auth token isn't valid or is not sent and it seems that using openmetrics_endpoint will not send it, whereas prometheus_url will 🤷

— Reply to this email directly, view it on GitHub https://github.com/DataDog/datadog-agent/issues/2667#issuecomment-1986793358, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACYZYTPGJSJHTYOTW6JWTYXLB4ZAVCNFSM4GD2CCJKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJYGY3TSMZTGU4A . You are receiving this because you were mentioned.Message ID: @.***>

[pmalek]

@grosser Do you think this should land in this repo or in https://github.com/DataDog/integrations-core/ ?

[grosser]

don't know ... use your best guess and owners will redirect ya :)