search

open-telemetry / opentelemetry-collector

OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
4.53k stars 1.48k forks source link

Options for Log Rotation in OpenTelemetry Collector Logging #10768

Open bvsvas opened 4 months ago

bvsvas commented 4 months ago

Describe the bug We are currently using the OpenTelemetry Collector to log telemetry data into a logfile. The logfiles are continuously growing, and we encounter issues when the pod gets restarted multiple times, causing us to lose logs.

We need guidance on the available options for log rotation and the best practices for implementing it.

Steps to reproduce

  1. Configure OTelCol to log telemetry data into a logfile.
  2. Observe that the logfile grows continuously.

What did you expect to see? We expect to have a mechanism for log rotation to manage the size of the logfiles and to ensure logs are not lost after pod restarts using with volume mounts.

What did you see instead? Logfile grows continuously.

What version did you use? v0.105.0

What config did you use?

service:
  telemetry:
    logs:
      level: debug
      output_paths: ${POD_NAME}-collector.log
      error_output_paths: ${POD_NAME}-collector-error.log
    metrics:
      address: ":8888"

Environment OpenTelemetry Collector version: v0.105.0 collector-telemetry

Additional context GitHub Issue https://github.com/open-telemetry/opentelemetry-collector/issues/1048 talks about logrotation.

What are the options for log rotation in OpenTelemetry Collector? What are the best practices to implement log rotation effectively?

bvsvas commented 4 months ago

label collector-telemetry