kmtong / logback-redis-appender

Logback Redis Appender
Apache License 2.0
114 stars 64 forks source link

Build Status

Usage

Now in Maven Central Repository:

<dependency>
  <groupId>com.cwbase</groupId>
  <artifactId>logback-redis-appender</artifactId>
  <version>1.1.6</version>
</dependency>

Configurable Options

Redis Related (RedisAppender Attributes)

Event Related (JSONEventLayout Attributes)

Since 1.1.1 these fields support MDC property resolution by @{varname}.

Logback/Java Specific (JSONEventLayout Attributes)

Note

Custom Layout

If you want to use other Layout (e.g. net.logstash.logback.layout.LogstashLayout) instead of our own JSONEventLayout, see the sample configuration below. (Since version 1.1.5)

Logging Asynchronously

As this appender would synchronously log to the Redis server, this may cause the logging thread to be hanged on some error conditions (network timeout or so). One resolution would be using the AsyncAppender provided by standard logback. Please refer to the below example configurations. (Thanks GuiSim for pointing this out)

Default values for MDC properties

MDC properties can be configured with default values by using the :- signifier. For example: @{varname:-foo} will result in foo if the varname property is not defined.

Example

Logback XML Configuration:

<appender name="LOGSTASH" class="com.cwbase.logback.RedisAppender">
    <source>mySource</source>
    <sourcePath>mySourcePath</sourcePath>
    <type>myApplication</type>
    <tags>production</tags>
    <host>192.168.56.10</host>
    <port>6379</port>
    <key>logstash</key>
</appender>

Logstash Configuration:

input {
 redis {
  codec => json
  host => "192.168.56.10"
  port => 6379
  key => "logstash"
  data_type => "list"
 }
}

Use with AsyncAppender:

<configuration>
  <appender name="LOGSTASH" class="com.cwbase.logback.RedisAppender">
    <source>mySource</source>
    <sourcePath>mySourcePath</sourcePath>
    <type>myApplication</type>
    <tags>production</tags>
    <host>192.168.56.10</host>
    <port>6379</port>
    <key>logstash</key>
  </appender>
  <appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
    <appender-ref ref="LOGSTASH" />
  </appender>
  <root level="DEBUG">
    <appender-ref ref="ASYNC" />
  </root>
</configuration>

Use Custom Layout:

<configuration>
  <appender name="LOGSTASH" class="com.cwbase.logback.RedisAppender">
    <!-- RedisAppender Attributes Here -->
    <host>192.168.56.10</host>
    <port>6379</port>
    <key>logstash</key>
    <!-- Use your own Custom Layout here -->
    <layout class="com.cwbase.logback.JSONEventLayout">
      <!-- JSONEventLayout Attributes Here -->
      <source>mySource</source>
      <sourcePath>mySourcePath</sourcePath>
      <type>myApplication</type>
      <tags>production</tags>
    </layout>
  </appender>
  <root level="DEBUG">
    <appender-ref ref="LOGSTASH" />
  </root>
</configuration>

ChangeLogs

Version 1.1.6 -> 1.1.5

Version 1.1.5 -> 1.1.4

Version 1.1.4 -> 1.1.2

Version 1.1.1 -> 1.1.2

Version 1.1.0 -> 1.1.1

Version 1.0.0 -> 1.1.0

Logstash has re-defined its JSON message format as [An event is simply a tuple of (timestamp, data).]

 {
   "@timestamp": "2013-02-09T20:39:26.234Z",
   "@version": "1",
   message: "hello world"
 }

This implies that the @-prefixed keys is not longer valid and allows a more flexible event data. The original mapping can be found at oldlogstashjson.rb file.

There is one unmapped field "@source". I will turn that into "source" field anyways.