grafana / k6

A modern load testing tool, using Go and JavaScript - https://k6.io
GNU Affero General Public License v3.0
26.03k stars 1.27k forks source link

K6_OTEL_EXPORTER_TYPE=http overrides K6_OTEL_GRPC_EXPORTER_ENDPOINT=prometheus:9090 #4070

Open agilob opened 5 hours ago

agilob commented 5 hours ago

Brief summary

Prometheus in version 2.54 has OTEL on http not GRPC, so to use it I'm using the following script command:

K6_OTEL_EXPORTER_TYPE=http K6_OTEL_GRPC_EXPORTER_ENDPOINT=prometheus:9090 K6_OTEL_HTTP_EXPORTER_URL_PATH=/api/v1/otlp/v1/metrics K6_OTEL_METRIC_PREFIX=agilob  ./k6 run --out experimental-opentelemetry main.js

k6 version

v0.55.0

OS

linux

Docker version and image (if applicable)

No response

Steps to reproduce the problem

but the script fails because defining K6_OTEL_EXPORTER_TYPE overrides my K6_OTEL_GRPC_EXPORTER_ENDPOINT:

❯ K6_OTEL_EXPORTER_TYPE=http K6_OTEL_GRPC_EXPORTER_ENDPOINT=prometheus:9090 K6_OTEL_HTTP_EXPORTER_URL_PATH=/api/v1/otlp/v1/metrics K6_OTEL_METRIC_PREFIX=agilob  ./k6 run --out experimental-opentelemetry main.js

         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

     execution: local
        script: main.js
        output: opentelemetry (http, https://localhost:4318/api/v1/otlp/v1/metrics)

ERRO[0000] can't shutdown OpenTelemetry metric provider  error="failed to upload metrics: Post \"https://localhost:4318/api/v1/otlp/v1/metrics\": dial tcp [::1]:4318: connect: connection refused"
     █ setup

Expected behaviour

.

Actual behaviour

But when I remove K6_OTEL_EXPORTER_TYPE, my exporter URL is correct, but obviously won't work:

❯ K6_OTEL_GRPC_EXPORTER_ENDPOINT=prometheus:9090 K6_OTEL_HTTP_EXPORTER_URL_PATH=/api/v1/otlp/v1/metrics K6_OTEL_METRIC_PREFIX=agilob ./k6 run --out experimental-opentelemetry main.js

         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

     execution: local
        script: main.js
        output: opentelemetry (grpc, prometheus:9090)

ERRO[0010] can't shutdown OpenTelemetry metric provider  error="failed to upload metrics: context deadline exceeded: rpc error: code = DeadlineExceeded desc = received context error while waiting for new LB policy update: context deadline exceeded"

Now, the url is correct, but protocol isn't.

On top of it, If I add https:// to the exporter URL, it starts producing #4060 :


         /\      Grafana   /‾‾/
    /\  /  \     |\  __   /  /
   /  \/    \    | |/ /  /   ‾‾\
  /          \   |   (  |  (‾)  |
 / __________ \  |_|\_\  \_____/

     execution: local
        script: main.js
        output: opentelemetry (grpc, https://prometheus:9090)

ERRO[0000] can't shutdown OpenTelemetry metric provider  error="failed to upload metrics: failed to exit idle mode: invalid target address https://prometheus:9090, error info: address https://prometheus:9090:443: too many colons in address"
agilob commented 4 hours ago

Played with it a bit more and have to say this gives funny outcome:

K6_OTEL_EXPORTER_TYPE=http K6_OTEL_GRPC_EXPORTER_INSECURE=false \
K6_OTEL_GRPC_EXPORTER_ENDPOINT=prometheus:9090 \
K6_OTEL_HTTP_EXPORTER_URL_PATH=/api/v1/otlp/v1/metrics \
K6_OTEL_METRIC_PREFIX=agilob

output: opentelemetry (http, https://localhost:4318/api/v1/otlp/v1/metrics)

Setting insecure exporter overrides exporter URL and protocol?

Yup, works as documented below: changes my protocol and exporter to the default :D

K6_OTEL_GRPC_EXPORTER_ENDPOINT  Configures the gRPC exporter endpoint. Default is localhost:4317.