search

danielwegener / logback-kafka-appender

Logback appender for Apache Kafka
Apache License 2.0
640 stars 263 forks source link

Do we lose logs if the brokers aren't available? #92

Open javadevmtl opened 4 years ago

javadevmtl commented 4 years ago

Just curious, if the brokers aren't available, do we eventually lose logs with this solution? For example using a log tailer like Filebeat or Fluent.D thos mecanism will keep track their place and retry as needed.

danielwegener commented 4 years ago

There is no automatism that would retain and retransmit logs that could not be delivered directly. You MAY use a fallback-appender (ref to readme) that would send log messages somewhere else where they might be stored and retransmitted later. Example: My original use case was to append logs from read-only containers to kafka. So as long as Kafka was available, the container would publish it's logs to kafka. If there was an broker outage, it's log messages would be sent to the fallback-appender. The fallback appender in this case would be a console.err appender that would be picked up by the docker logging driver and act as a fallback.