search

elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
69 stars 3.5k forks source link

add event data to log4j layout pattern #12472

Open gbanasiak opened 3 years ago

gbanasiak commented 3 years ago

To make troubleshooting easier we could expose event data in log4j thread context:

(ref)

appender.rolling.layout.pattern = \
  [%d{ISO8601}][%-5p][%-25c]%notEmpty{[%X{pipeline.id}]}%notEmpty{[%X{plugin.id}]}{EVENTDATA} %m%n
                                                                                  ^^^

Today we add pipeline.id and plugin.id when the thread starts (here). Could we change that to per-event for selected event data?

andsel commented 3 years ago

pipeline.id and plugin.id exists before the event is created, bring as example an input plugin, which reads data from network and as last action create the event to be push on the queue before filters/output processing. In this case for some logs in the input there's no concept of event. In other cases like filters, which works with batch of events, could we little hard to set the Logstash event into the log4j context, since the plugin works on the batch as a whole from the Logstash perspective.