logstash-plugins / logstash-output-kafka

Kafka Output for Logstash
Apache License 2.0
74 stars 76 forks source link

log4j warnings for kafka #118

Open tokred opened 7 years ago

tokred commented 7 years ago

When starting my Logstash instance, I get the following warnings regarding log4j and Kafka output in my /var/log/messages. I can't find any other Kafka output related entries in the logs after this:

Feb 13 16:36:38 log-shipper01 systemd: Started logstash.
Feb 13 16:36:38 log-shipper01 systemd: Starting logstash...
Feb 13 16:36:59 log-shipper01 logstash: Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
Feb 13 16:37:01 log-shipper01 logstash: log4j:WARN No appenders could be found for logger (org.apache.kafka.clients.producer.ProducerConfig).
Feb 13 16:37:01 log-shipper01 logstash: log4j:WARN Please initialize the log4j system properly.
Feb 13 16:37:01 log-shipper01 logstash: log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Feb 13 16:37:01 log-shipper01 logstash: Exception in thread "[main]>worker0" java.lang.ClassCastException: org.logstash.bivalues.StringBiValue cannot be cast to java.lang.String
Feb 13 16:37:01 log-shipper01 logstash: at org.logstash.Event.toString(Event.java:312)
Feb 13 16:37:01 log-shipper01 logstash: at org.logstash.ext.JrubyEventExtLibrary$RubyEvent.ruby_to_s(JrubyEventExtLibrary.java:209)
Feb 13 16:37:01 log-shipper01 logstash: at org.logstash.ext.JrubyEventExtLibrary$RubyEvent$INVOKER$i$0$0$ruby_to_s.call(JrubyEventExtLibrary$RubyEvent$INVOKER$i$0$0$ruby_to_s.gen)
Feb 13 16:37:01 log-shipper01 logstash: at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
Feb 13 16:37:01 log-shipper01 logstash: at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
Feb 13 16:37:01 log-shipper01 logstash: at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
Feb 13 16:37:01 log-shipper01 logstash: at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
Feb 13 16:37:01 log-shipper01 logstash: at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
Feb 13 16:37:01 log-shipper01 logstash: at org.jruby.ast.IfNode.interpret(IfNode.java:118)
Feb 13 16:37:01 log-shipper01 logstash: at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
Feb 13 16:37:01 log-shipper01 logstash: at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
[...]

Not sure if the ClassCastException is related, I posted it already on the Logstash repo (https://github.com/elastic/logstash/issues/6695).

My log4j2.properties is default as shipped with the RPM:

status = error
name = LogstashPropertiesConfig

appender.rolling.type = RollingFile
appender.rolling.name = plain_rolling
appender.rolling.fileName = ${sys:ls.logs}/logstash-${sys:ls.log.format}.log
appender.rolling.filePattern = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}.log
appender.rolling.policies.type = Policies
appender.rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.rolling.policies.time.interval = 1
appender.rolling.policies.time.modulate = true
appender.rolling.layout.type = PatternLayout
appender.rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %-.10000m%n

appender.json_rolling.type = RollingFile
appender.json_rolling.name = json_rolling
appender.json_rolling.fileName = ${sys:ls.logs}/logstash-${sys:ls.log.format}.log
appender.json_rolling.filePattern = ${sys:ls.logs}/logstash-${sys:ls.log.format}-%d{yyyy-MM-dd}.log
appender.json_rolling.policies.type = Policies
appender.json_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.json_rolling.policies.time.interval = 1
appender.json_rolling.policies.time.modulate = true
appender.json_rolling.layout.type = JSONLayout
appender.json_rolling.layout.compact = true
appender.json_rolling.layout.eventEol = true

rootLogger.level = ${sys:ls.log.level}
rootLogger.appenderRef.rolling.ref = ${sys:ls.log.format}_rolling
svrc commented 7 years ago

see https://github.com/logstash-plugins/logstash-output-kafka/pull/134 which was my resolution