grafana / alloy

OpenTelemetry Collector distribution with programmable pipelines
https://grafana.com/oss/alloy
Apache License 2.0
1.43k stars 211 forks source link

Faro Receiver - Propagate certain headers to measurements #1538

Open justinbwood opened 2 months ago

justinbwood commented 2 months ago

Request

I'd like to be able to grab certain headers from the incoming Faro payload and insert them into the logfmt key-value pairs generated when Faro creates a log line for measurements.

Maybe this could include log, event, and exception types too, but I don't see that as being particularly useful.

Use case

I'd like to include certain viewer request headers on measurements, like latitude and longitude. By including this metadata on measurements, we could graph page load times by geographic region using Grafana's geomap panel.

We can enable include_metadata in the Faro receiver, but that only propagates connection metadata to the downstream otelcol.Consumer for traces. Example of adding latitude and longitude to traces:

faro.receiver "faro" {
  server {
    include_metadata = true
    // other receiver config
  }

  output {
    logs   = [ /* ... */ ]
    traces = [otelcol.processor.attributes.headers.input]
  }
}

otelcol.processor.attributes "headers" {
  action {
    key = "http.client.latitude"
    from_context = "metadata.cloudfront-viewer-latitude"
    action = "upsert"
  }

  action {
    key = "http.client.longitude"
    from_context = "metadata.cloudfront-viewer-longitude"
    action = "upsert"
  }

  output {
    traces = [otelcol.exporter.otlphttp.traces.input]
  }
}

otelcol.exporter.otlphttp "traces" {
  client {
    endpoint = "https://tempo:443"
  }
}

One idea for measurements would be an optional config block in faro.receiver that could map headers to logfmt fields:

faro.receiver "faro" {
  server {
    include_metadata = true
    // other receiver config
  }

  headers {
    mapping = {
      latitude = "cloudfront-viewer-latitude",
      longitude = "cloudfront-viewer-longitude",
    }
  }

  output {
    logs   = [loki.process.faro.receiver]
  }
}

loki.process "faro" {
  stage.logfmt {
    mapping = {
      latitude = "",
      longitude = "",
    }
  }

  stage.structured_metadata {
    values = {
      latitude = "",
      longitude = "",
    }
  }

  forward_to = [loki.write.logs.receiver]
}

loki.write "logs" {
  endpoint {
    url = "https://loki:443"
  }
}
github-actions[bot] commented 1 month ago

This issue has not had any activity in the past 30 days, so the needs-attention label has been added to it. If the opened issue is a bug, check to see if a newer release fixed your issue. If it is no longer relevant, please feel free to close this issue. The needs-attention label signals to maintainers that something has fallen through the cracks. No action is needed by you; your issue will be kept open and you do not have to respond to this comment. The label will be removed the next time this job runs if there is new activity. Thank you for your contributions!