Feat: Allow for specifying multiple OTLP Headers (see `OTEL_EXPORTER_OTLP_HEADERS`)

[SeanHood]

Feature Request

Hey, thought I'd take this new plugin for a spin but ran into a limitation not being able to specify multiple headers.

We use Honeycomb.io, who support OTLP/GRPC natively. However you need to specify the following headers:

x-honeycomb-team: YOUR_API_KEY
x-honeycomb-dataset: YOUR_DATASET

Currently it looks like only a single header can be specified. Using the otel-collector will be a reasonable workaround for us for now.

[cyrille-leclerc]

Thanks @SeanHood for using this integration. Could Honeycomb support passing the dataset name as an attribute of the traces and metrics? We identified the need for a generic mechanism to add such attributes, see https://github.com/jenkinsci/opentelemetry-plugin/issues/86 .

[SeanHood]

I think they're probably set on using the headers for this configuration. It would be similar to your linked issue, but exposing the option to set multiple OTEL_EXPORTER_OTLP_HEADERS: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md Looks like Honeycomb maybe the only one who require two headers.

[cyrille-leclerc]

Good catch for OTEL_EXPORTER_OTLP_HEADERS. I have used it to instrument applications and forgot about it when working on Jenkins instrumentation :-) Totally makes sense.

[dcrowe]

For those who come along later, here's the otel-collector config that worked for me:

# Based on Honeycomb tutorial:
# https://docs.honeycomb.io/getting-data-in/opentelemetry/otel-collector/

receivers:
  otlp:
    protocols:
      grpc: # on port 55680
      http: # on port 55681

processors:
  batch:

exporters:
  logging:
    loglevel: warn
  otlp/honeycomb:
    endpoint: "api.honeycomb.io:443"
    headers:
      "x-honeycomb-team": "..."
      "x-honeycomb-dataset": "..."

extensions:
  health_check:
  pprof:
  zpages:

service:
  extensions: [health_check, pprof, zpages]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, otlp/honeycomb]
    metrics:
      receivers: [otlp]
      exporters: [logging]

And here's a working Trace visualisation URL template: https://ui.honeycomb.io/max-mine-iot/datasets/jenkins-builds/trace?trace_id=${traceId}&trace_start_ts=${startTime.epochSecond-60}

[cyrille-leclerc]

Solved in 0.22 supporting the OTEL_EXPORTER_OTLP_HEADERS environment variable and the -Dotel.exporter.otlp.headers jvm property

[cyrille-leclerc]

Can now also be defined in the "Configuration Properties" text area

[RishiRGupta]

Hey @cyrille-leclerc to specify multiple values of OTLP headers , do we simply do something like OTEL_EXPORTER_OTLP_HEADERS="x-property-1=value1,x-property-2=value-2" this in the configuration properties ? Also is there a way to test if these properties are being added in the traces / logs / metrics ?

[cyrille-leclerc]

Did you try specifying a config property otel.exporter.otlp.headers=... in the advanced section of the plugin config section?