This PR contains a sample configuration for the OpenTelemetry collector to parse and forward Nginx Ingress Controller Access logs. A Kibana dashboard is also provided.
Decisions changelog
A custom datastream.dataset attribute (nginx_ingress_controller.access) has been set, the reasoning behind it is to have a different datastream for each receiver/integration.
Even some attributes are not yet defined in SemConv, an approximate structure has been used (e.g http.request.id). Otel structure in favor of ECS.
The GeoIP processor is disabled as it requires a local database, uncomment its configuration with a valid database path if needed.
The sample configuration uses resourcedetection/system processor to retrieve and add the host.name attribute, which can be used for filtering in the dashboard.
Missing charts
Operating System breakdown (No OS UserAgent attributes with OTTL)
This PR contains a sample configuration for the OpenTelemetry collector to parse and forward Nginx Ingress Controller Access logs. A Kibana dashboard is also provided.
Decisions changelog
ExtractGrockPattern
has been used instead of regex, the configuration became less error prone and easier to align with Elastic's current pipeline: https://github.com/elastic/integrations/blob/main/packages/nginx_ingress_controller/data_stream/access/elasticsearch/ingest_pipeline/default.ymlresourcedetection/system
processor to retrieve and add thehost.name
attribute, which can be used for filtering in the dashboard.Missing charts
upstream.response.time
is computed by the sum of all the responses. All the responses are captured in an array attribute, but there is no OTTL function to sum all the values (e.gupstream.response.time: Slice(["0.002", "0.001"])
). See Elastic integration https://github.com/elastic/integrations/blob/main/packages/nginx_ingress_controller/data_stream/access/elasticsearch/ingest_pipeline/default.yml#L111