[FR] Add authorization with an API KEY token

[qmonitoring]

Use case: I am trying to scrape an endpoint with an API KEY аuthorization using Prometheus with json-exporter ( prom/prometheus:main and prometheuscommunity/json-exporter:v0.3.0 docker images):

docker run -d --network=host --name=prometheus -v /path/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus:main --config.file=/etc/prometheus/prometheus.yml --web.listen-address=:9095

docker run -d --name=json-exporter --network=host -v /path/jsonexporter/config.yml:/config.yml prometheuscommunity/json-exporter:v0.3.0 --config.file=/config.yml

with the following Prometheus prometheus.yml configuration file:

global:
  evaluation_interval: 15s
  scrape_interval: 15s
  scrape_timeout: 10s
  external_labels:
    environment: prometheus_in_docker

scrape_configs:

  - job_name: json_exporter_self
    static_configs:
      - targets:
        - <host>:7979 ## Location of the json exporter's real <hostname>:<port>

  ## gather the metrics from third party json sources, via the json exporter
  - job_name: json_exporter_data
    metrics_path: /probe
    authorization:
      credentials: '{my_secret_token}'
      type: apikey  
    static_configs:
      - targets:
        - http://<endpoint_host>:<port>/api/v1/data
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: <host>:7979 ## Location of the json exporter's real <hostname>:<port>

and the following json-exporter config.yml configuration file:

metrics:
- name: example_metric
  path: "{ .data }"
  help: Example 
  labels:
    environment: testdata # static label

prom/prometheus:main and prometheuscommunity/json-exporter:v0.3.0 are up and running, self scraping works fine (http://:7979/metrics), but a json scraping doesn't work (http://:7979/probe), the error is the following:

_Failed to fetch JSON response. TARGET: http://:/api/v1/data, ERROR: 401 Unauthorized_

The endpoint itself is available: _curl -X GET 'http://:/api/v1/data' -H 'Authorization: {my_secrettoken}'

Feature request: Could you please implement authorization with API KEY token?

(please see link with additional discussion in the Prometheus repository)

[busa-blade]

Does anyone know if this has been implemented?

[alinix-alinix]

Does anyone know if this has been implemented?

I have a similar issue. Trying to scrape json path via api url like http://MyHost/platform_api/GetSomeStuff/?account_id=123456&api_key=11112222233334444 It returns json in browser or curl just fine. But i have no success probing that url with json_exporter http://192.168.9.103:7979/probe?target=http://MyHost/platform_api/GetSomeStuff/?account_id=123456&api_key=11112222233334444 It seems to pass an account id but no api_key, so i'm getting authorization error.

[dahoat]

I have a similar issue. If you run json_exporter locally and use Wireshark, you can see that the exporter only forwards the first get argument.

[dahoat]

Sorry I didn't wait with my last comment until I knew the solution:

Solution

Replace & with %26 because you have to escape your &.

Root cause

If you request http://exporter.local/probe?target=someurl&apiKey=abcd then apiKey is delivered as GET parameter to probe and is not read as part of the string intended for target.
Therefore, request http://exporter.local/probe?target=someurl%26apiKey=abcd

Code location

https://github.com/prometheus-community/json_exporter/blob/d1456b89177b93ed15ef6fc43dc0b4c529599c3c/cmd/main.go#L110

[acondrat]

I think this is already supported with the http_client_config. Look at the example provided in https://github.com/prometheus-community/json_exporter/blob/master/examples/config.yml

I needed something similar in a Kubernetes environment and my config looks like

          http_client_config:
            authorization:
              type: Key
              credentials_file: /tmp/api-key.txt

Translated to curl this would be something like

curl -H "Authorization: Key MySecretApiKey" "exporter:7979/probe?target=http://mytarget"

For other options see https://pkg.go.dev/github.com/prometheus/common/config#Authorization

[urpylka]

We need to have ability to add custom headers and hide it. For example gitlab api provide to use header field - token to authentificate. Yes, maybe gitlab team made bad solution, but, for now, I don't ability to use secure.

[morremeyer]

Authorization with the standardized HTTP Authorization methods is possible as @acondrat described in https://github.com/prometheus-community/json_exporter/issues/87#issuecomment-1305469450.

For example, the following module configuration sets the header Authorization: Bearer 123456 when the file /credentials/some_app.txt contains the string 123456.

modules:
  some_application:
    http_client_config:
      authorization:
        credentials_file: /credentials/some_app.txt

I don't think json_exporter should support non-standard-conformant authentication methods since this would add maintenance overhead that is not rectifiable - any application should support at least one of the standardized methods of Bearer tokens, Basic auth, or OAuth2.

[schoen2]

Hi @morremeyer,

I hope you can help me out. I have deployed Json-Exporter and Prometheus as a Helm chart in my Kubernetes cluster.

I have an API query that I can successfully submit in the terminal and get a return:

curl -X GET https://api.viessmann.com/iot/v2/features/installations/ID/gateways/ID/devices/ID/features -H "Authorization: Bearer <token>“

If I go by the http_client_config documentation (https://pkg.go.dev/github.com/prometheus/common/config#HTTPClientConfig) bearer_token or authorization.credentials must be of type Secret. Unfortunately, I can't get the bearer token to work properly as a Kubernetes secret.

This is what my json exporter config looks like:

configuration:
  config: |
    ---
    modules:
      some_application:
        metrics:
        - name: viessmann
          type: object
          path: '{ .data.0 }'
          help: viessmann
          labels:
            id: "viessmann"
            name: "viessmann"
          values:
            apiVersion: '{.apiVersion}'
        http_client_config:
          bearer_token: 'viessmann-secret'
or
        http_client_config:
          authorization:
            credentials: 'viessmann-secret'

The Kubernetes-secret I create like:

apiVersion: v1
kind: Secret
metadata:
  name: viessmann-secret
  namespace: monitoring
type: Opaque
data:
  bearer_token: BASE64_ENCODED_TOKEN_VALUE_HERE

I hope you or someone else can help me :/ Thanks in advance

[morremeyer]

Hey @schoen2, let me post the full configuration that we use.

The http_client_config.bearer_token or http_client_config.authorization.credentials need to be the literal token, not a reference to a secret with the token.

If you want to read the token from a secret, which I would recommend, you'll need to mount the secret as a file and reference that with http_client_config.authorization.credentials_file.

To illustrate, let me add the config we have deployed currently.

Deploying with helm for renovate-ce

We're using https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus-json-exporter in version 0.11.0 to deploy.

The values we have configured in the values file

```yaml serviceMonitor: enabled: true targets: - name: renovate-ce url: http://the-endpoint-for-renovate module: renovate configuration: config: | --- modules: renovate: http_client_config: authorization: credentials_file: /credentials/renovate/renovate-ce.txt metrics: - name: renovate_organization_count valuetype: gauge path: "{ .app.organizationCount }" help: The number of organizations known to renovate - name: renovate_repository_count valuetype: gauge path: "{ .app.repositoryCount }" help: The number of repositories known to renovate - name: renovate_jobs_processed valuetype: counter path: "{ .jobs.history.processed }" help: The number of jobs processed since the instance has started - name: renovate_queue_size valuetype: gauge path: "{ .jobs.queue.size }" help: The number of jobs currently in the queue additionalVolumes: - name: credentials-renovate-ce secret: secretName: credentials-renovate-ce additionalVolumeMounts: - name: credentials-renovate-ce mountPath: /credentials/renovate/renovate-ce.txt subPath: mendRnvServerApiSecret ```

The secret

```yaml apiVersion: v1 kind: Secret metadata: name: credentials-renovate-ce namespace: kube-prometheus-stack type: Opaque data: mendRnvServerApiSecret: Tm8gY2hhbmNlLCBJIHJlcGxhY2VkIHRoaXMgYmVmb3JlIHBhc3RpbmcgaXQ= ```

[schoen2]

Great!! Thanks a lot! I don't know the difference between credentials_file and credentials about the reference to a secret.

[luis15pt]

@morremeyer can you help ? running into a small issue while trying to use json-exporter with an api endpoint that uses an api_key, no matter what i try i end up with 401 Unauthorized.

This is the working format in curl:

curl -X GET https://example.com/v1/core/images -H 'api_key: xxxxxxxxxxxxxxxxxxx' When using it over the json-exporter http://192.168.7.250:7979/probe?module=default&target=https%3A%2F%2Fexample.com%2Fv1%2Fcore%2Fimages

Failed to fetch JSON response. TARGET: https://example.com/v1/core/images, ERROR: 401 Unauthorized This is my config file, am i missing something?

modules:
  default:
  http_client_config:
    follow_redirects: true
    enable_http2: true
    tls_config:
      insecure_skip_verify: true
    http_headers:
      api_key: 'xxxxxxxxxxxxxxxxxxx'
    metrics:
      - type: gauge
        name: image_name
        help: "Image Names"
        path: $.images[*].images[*].name
        labels:
          image_name: $.name

Ref: https://pkg.go.dev/github.com/prometheus/common/config#HTTPClientConfig

[morremeyer]

@luis15pt Can you link to the API documentation? If not, are you sure the api key has to be sent this way? That is not a standardized HTTP header.

I would expect something like „Authorization: Bearer xxxx“

[luis15pt]

I will double check but currently its documented only as api_key, just noticed maybe i need to add the "accept: application/json" ?

curl -X GET "https://infrahub-api.nexgencloud.com/v1/pricebook" \ -H "accept: application/json"\ -H "api_key: YOUR API KEY"

https://infrahub-doc.nexgencloud.com/docs/api-reference/pricebook-resources/pricebook

On Mon, Jun 24, 2024 at 10:01 PM morre @.***> wrote:

@luis15pt https://github.com/luis15pt Can you link to the API documentation? If not, are you sure the api key has to be sent this way? That is not a standardized HTTP header.

I would expect something like „Authorization: Bearer xxxx“

— Reply to this email directly, view it on GitHub https://github.com/prometheus-community/json_exporter/issues/87#issuecomment-2187394880, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYCEIF5MVAHWUBEAV5HIWM3ZJCCLBAVCNFSM4YO5L3QKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJYG4ZTSNBYHAYA . You are receiving this because you were mentioned.Message ID: @.***>