search

danielwegener / logback-kafka-appender

Logback appender for Apache Kafka
Apache License 2.0
640 stars 263 forks source link

Write to multiple topics #81

Closed saithala closed 5 years ago

saithala commented 5 years ago

My requirement is to log to different/multiple topics. I see a note in the read me that advises to create one appender for each topic. My question is how to dynamically/programatically chose the appender while logging?

zengchengjie commented 5 years ago

I got same question too

danielwegener commented 5 years ago

I am not sure if really I understand the question. You either log your message to a logger that is tied to a certain appender (=topic) 

<logger name="com.org.package1" level="INFO"/>
   <appender-ref ref="KafkaAppenderForTopic1" />
</logger>
<logger name="com.org.package2" level="INFO"/>
   <appender-ref ref="KafkaAppenderForTopic1" />
</logger>

or you configure multiple appenders with different topics for the same logger.

 <root level="INFO">
    <appender-ref ref="KafkaAppenderForTopic1" />
    <appender-ref ref="KafkaAppenderForTopic2" />
  </root>

If you are looking for a mechanism to select an appender based on the content of the log-message itself, well, that's not quite what logback is designed for. There is something called "marker" in the slf4j api that could be (ab)used and one could create an "delegate appender" that routes log messages to different appenders based on its content (i.e. your tags). But AFAIK there is nothing out of the box provided by logback nor this plugin.

saithala commented 5 years ago

Thanks for your response. Going by your example, I have two appenders, one for each topic.

<root level="INFO">
    <appender-ref ref="KafkaAppenderForTopic1" />
    <appender-ref ref="KafkaAppenderForTopic2" />
  </root>

Now, when I use log.info(msg), I wanted to know if there is a way to chose the appender to write using KafkaAppenderForTopic1 or KafkaAppenderForTopic2.

danielwegener commented 5 years ago

As described above, that's not a usual use case for a logging framework like slf4j, but you could use something like Logger.info(marker, string) to mark a log message and then you could build a custom logback appender that would distribute logging events between nested appenders (your Topic appenders from above) based on the presence of Markers. But this is probably something the logback community can give a better answer on (the problem is not specific to this kafka-appender).

zengchengjie commented 5 years ago

thank U for response，I should put appender to logger not root label ，its a low level error hahaha