search

logstash-plugins / logstash-integration-kafka

Kafka Integration for Logstash, providing Input and Output Plugins
Apache License 2.0
32 stars 60 forks source link

docs: clarify limitations of header decoration #173

Open yaauie opened 2 months ago

yaauie commented 2 months ago

Two known limitations with headers:

  1. Only headers whose values are utf-8 byte sequences will be added to the event (impl). While in theory this could change in the future, receiving anything other than a known-constant encoding is problematic for a variety of reasons and likely the reason for the current implementation (e.g., not being embeddable in JSON without loss, which precludes sending the raw bytes to Elasticsearch or other outputs).
  2. Kafka allows multiple headers to share a key, but as-implemented we will overwrite a key if we observe and set it twice.