search

mp911de / logstash-gelf

Graylog Extended Log Format (GELF) implementation in Java for all major logging frameworks: log4j, log4j2, java.util.logging, logback, JBossAS7 and WildFly 8-12
http://logging.paluch.biz
MIT License
424 stars 101 forks source link

get the full stacktrace on wildfly #64

Closed kgoedert closed 8 years ago

kgoedert commented 8 years ago

Hi,

I just set up the wildfly 9 module to send the logs to graylog2. It worked. But my stack trace appears to be cut. Is there a way I can format the output to see the full stack trace, with the error and all the caused by clauses?

mp911de commented 8 years ago

Could you post your logger config? kgoedert notifications@github.com schrieb am Mo., 1. Feb. 2016 um 18:34:

Hi,

I just set up the wildfly 9 module to send the logs to graylog2 It worked But my stack trace appears to be cut Is there a way I can format the output to see the full stack trace, with the error and all the caused by clauses?

— Reply to this email directly or view it on GitHub https://github.com/mp911de/logstash-gelf/issues/64.

kgoedert commented 8 years ago 
<custom-handler name="GelfLogger" class="biz.paluch.logging.gelf.wildfly.WildFlyGelfLogHandler" module="biz.paluch.logging">
    <level name="INFO" />
    <properties>
        <property name="host" value="udp:10.1.1.27" />
        <property name="port" value="12201" />
        <property name="version" value="1.1" />
        <property name="facility" value="java-test" />
        <property name="extractStackTrace" value="true" />
        <property name="mdcProfiling" value="true" />
        <property name="timestampPattern" value="yyyy-MM-dd HH:mm:ss,SSSS" />
        <property name="maximumMessageSize" value="15000" />

        <!-- This are static fields -->
        <property name="additionalFields" value="fieldName1=fieldValue1,fieldName2=fieldValue2" />
        <!-- Optional: Specify field types -->
        <property name="additionalFieldTypes" value="fieldName1=String,fieldName2=Double,fieldName3=Long" />

        <!-- This are fields using MDC -->
        <property name="mdcFields" value="mdcField1,mdcField2" />
        <property name="dynamicMdcFields" value="mdc.*,(mdc|MDC)fields" />
        <property name="includeFullMdc" value="true" />
    </properties>
</custom-handler>

I also tried with filterStackTrace false.

mp911de commented 8 years ago

filterStackTrace compresses the stack trace so framework stack lines can be combined. The whole chain of exceptions is preserved. The config looks good. You could do two things:

  1. There's a custom log formatter (http://logging.paluch.biz/examples/wildfly-json.html). You could add another file appender to log the data into a log file to check whether logstash-gelf or graylog truncates the stack trace
  2. Can you post the stack trace how it's displayed within graylog?
kgoedert commented 8 years ago

This is what I expected to see

16:07:50,112 ERROR [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0055: Caught exception during boot: org.jboss.as.controller.persistence.ConfigurationPersistenceException: WFLYCTL0085: Failed to parse configuration
    at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:131)
    at org.jboss.as.server.ServerService.boot(ServerService.java:350)
    at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:271)
    at java.lang.Thread.run(Thread.java:745)
Caused by: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module org.jboss.as.jsf
    at org.jboss.as.controller.parsing.ExtensionXml.parseExtensions(ExtensionXml.java:155)
    at org.jboss.as.server.parsing.StandaloneXml.readServerElement_1_4(StandaloneXml.java:433)
    at org.jboss.as.server.parsing.StandaloneXml.readElement(StandaloneXml.java:144)
    at org.jboss.as.server.parsing.StandaloneXml.readElement(StandaloneXml.java:106)
    at org.jboss.staxmapper.XMLMapperImpl.processNested(XMLMapperImpl.java:110)
    at org.jboss.staxmapper.XMLMapperImpl.parseDocument(XMLMapperImpl.java:69)
    at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:123)
    ... 3 more
Caused by: java.util.concurrent.ExecutionException: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at org.jboss.as.controller.parsing.ExtensionXml.parseExtensions(ExtensionXml.java:147)
    ... 9 more
Caused by: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module
    at org.jboss.as.controller.parsing.ExtensionXml.loadModule(ExtensionXml.java:196)
    at org.jboss.as.controller.parsing.ExtensionXml.access$000(ExtensionXml.java:69)
    at org.jboss.as.controller.parsing.ExtensionXml$1.call(ExtensionXml.java:127)
    at org.jboss.as.controller.parsing.ExtensionXml$1.call(ExtensionXml.java:124)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: org.jboss.modules.ModuleNotFoundException: com.sun.jsf-impl:main
    at org.jboss.modules.Module.addPaths(Module.java:1042)
    at org.jboss.modules.Module.link(Module.java:1398)
    at org.jboss.modules.Module.relinkIfNecessary(Module.java:1426)
    at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:238)
    at org.jboss.as.controller.parsing.ExtensionXml.loadModule(ExtensionXml.java:178)
    ... 8 more

16:07:50,128 FATAL [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0056: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.

This is what I got in graylog

Exception: org.jboss.as.controller.persistence.ConfigurationPersistenceException: WFLYCTL0085: Failed to parse configuration Caused by: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module org.jboss.as.jsf Caused by: java.util.concurrent.ExecutionException: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module Caused by: javax.xml.stream.XMLStreamException: WFLYCTL0083: Failed to load module Caused by: org.jboss.modules.ModuleNotFoundException: com.sun.jsf-impl:main at org.jboss.modules.Module.addPaths(Module.java:1042) 7 lines skipped for [org.jboss] at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 1 line skipped for [org.jboss]
mp911de commented 8 years ago

Thanks. The line from graylog2 runs indeed through the stack trace filter (7 lines skipped for [org.jboss]). Disabling filterStackTrace will preserve the full trace. The other issue is that line breaks are scrambled up by graylog. A past issue (#46) indicated that this is caused by graylog somehow.

kgoedert commented 8 years ago

Sorry, if I did not understand your answer correctly, but as I said before, I did disable the filterStackTrace, but the log keeps appearing as I posted. Should I assume this is a graylog problem? Maybe the version I am using?

mp911de commented 8 years ago

Hmm, not sure. WildFly tends to keep certain config parameters. It's possible that the filterStackTrace property is still set if you disabled the filterStackTrace flag by removing the property from the XML config. Check the logging.properties file within the configuration directory of your server which properties are configured for the GelfLogger.

kgoedert commented 8 years ago

Editing the logging.properties files solved the problem. Thanks

mp911de commented 8 years ago

Thanks for your feedback. That was actually my last bet, glad it worked out.