[opentelemetry] Container Logshipping

[timojohlo]

Context

We chose OpenSearch as the Sink for OpenTelemetry (OTel) Logs. In order to digest logs accordingly we need to structure, enhance and process incoming container logs.

Acceptance Criteria

Log Collector:

• [x] Activate Daemonset for OTel to collect logs across different nodes
• [ ] FluentBit OpenTelemetry Format (OTLP) to OpenTelemetry Collector and output to OpenSearch
• [x] OpenTelemetry Collector Filelog output to OpenSearch
• [ ] Structure and enrich logs based on OTel format
  • according to https://opentelemetry.io/docs/specs/otel/logs/data-model-appendix/

References

148

[timojohlo]

The following task was not completed in Sprint 07/24

• [ ] FluentBit OpenTelemetry Format (OTLP) to OpenTelemetry Collector and output to OpenSearch

This task is still interesting as a use case to evaluate receiving and collecting logs in our infrastructure. We will look into it in the next sprints and evaluate further