open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
2.71k stars 2.14k forks source link

[pkg/stanza] Option for setting max log size in syslog parser #33777

Open bacherfl opened 3 days ago

bacherfl commented 3 days ago

Description: add MaxLogSize parameter to syslog parser. Note that for this option to be available, enable_octet_counting needs to be set to true, as this is an option that is exclusive to the octetcounting parser in the syslog library.

One aspect where I'm not sure about yet is regarding the placement of the max_log_size option: Right now, this option is also set within the TCP input configuration, whereas this new option would be one layer above, i.e. in the syslog base config. This would mean that this option could potentially be set to different values in the parser and tcp input config like for example:

receivers:
  syslog:
    protocol: rfc5424
    enable_octet_counting: true
    max_log_size: 200000000 # 200MiB
    tcp:
      listen_address: :4278
      max_log_size: 100000000 # 100MiB
exporters:
  debug:
service:
  pipelines:
    logs:
      receivers: [syslog]
      exporters: [debug]

For now I have implemented this in a way where if nothing is set if the tcp input config, the max_log_size value from the syslog base config will be used. If set in the tcp config, the tcp input will use that more specific value. To me this makes the most sense right now, but I appreciate any feedback on this.

Link to tracking Issue: #33182

Testing: so far added unit test for the syslog parser, will also add some tests for the syslog input config to test the behavior described above.

Documentation: TODO, will add once we have figured out all open questions