The message attribute in the JSON log has backslash, new line and special charecters

Hi Team,

I have applied the logstash-logback-encoder and the logs generate in JSON format but some portion of the logs as below for message attritbute still has backslash, new line and special charecters in it.

{"@timestamp":"2018-01-31T19:21:27.624+04:00","@version":1,"message":"{\"type\":\"API_REQUEST\",\"uid\":\"35f53fe5-7c63-4936-a617-14601073a5f7\",\"sessionId\":\"9E998B68C5A6D66C30D215DE0B4B5EF3\",\"remoteAddress\":\"0:0:0:0:0:0:0:1\",\"methodType\":\"POST\",\"headers\":\"{content-length=76, accept-language=en-US,en;q=0.9, cookie=JSESSIONID=50D17B32E96A6A8850F2A5FD52A4C07D, postman-token=a5be9806-a0a6-3905-e45d-2bea9036f3ed, origin=chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop, accept=*/*, host=localhost:3080, connection=keep-alive, content-type=application/json, cache-control=no-cache, accept-encoding=gzip, deflate, br, user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36}\",\"uri\":\"/user/login\",\"query\":null,\"contentType\":\"application/json\",\"requestTime\":\"2018-01-31 19:21:26.024\",\"responseTime\":\"2018-01-31 19:21:27.604\",\"executionTimeMillisecond\":1580,\"requestPayload\":\"{\\n \\\"\\": \\\"+971\\\",\\n \\\"phone\\\": \\\"545901009\\\",\\n \\\"employeeType\\\": \\\"PHONE\\\"\\n }\",\"responsePayload\":\"{\\\"status\\\":\\\"FAIL\\\",\\\"statusCode\\\":1000,\\\"message\\\":\\\"Internal Server Error\\\",\\\"desc\\\":\\\"FAIL\\\",\\\"debug\\\":\\\"org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalArgumentException: non null key required\\\\n\\\\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:982)\\\\n\\\\tat org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)\\\\n\\\\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:648)\\\\n\\\\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)\\\\n\\\\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:729)\\\\n\\\\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)\\\\n\\\\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)\\\\n\\\\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)\\\\n\\\\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)\\\\n\\\\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)\\\\n\\\\tat

Hi @SawyerFord

It looks like your application code is pre-converting an object to a JSON string, and then passing that string to the logger as the log message. For example, it looks like you are doing something like this:

logger.info("{\"type\":\"API_REQUEST\",...")

This is not the expected use case for using the encoders in logstash-logback-encoder. The encoders in logstash-logback-encoder expect normal (non-JSON formatted) messages to be logged by the application. The encoder itself then converts the log event to a JSON string for logging.

Since the encoder includes the message from the log event in a JSON field named message, it must escape the message string.

In other words... don't pre-convert your messages to JSON before you log them. If you're pre-converting your messages to JSON, you don't really even need to use the encoders from logstash-logback-encoder

If you have an object that you would like to be merged into the JSON event created by the encoders in logstash-logback-encoder, then you can use event-specific custom fields

Hi @philsttr

Below is the logger which already parses the message attribute in JSON. It is made to store inside GrayLogs.

<logger name="com.json.logging.config.log.JSONLogFilter" level="DEBUG" > <appender-ref ref="GELF" /> </logger>

Do I get this same message field inside the log output without encoding and I would also like to remove some of the new link charecters of the subfields of message JSON

What is com.json.logging.config.log.JSONLogFilter?

@philsttr : This is a filter extends the OncePerRequestFilter and has the the below logic

  ObjectNode obj=new ObjectNode();
  obj.put(''type","API_REQUEST")
  obj.put("uid", requestWrapper.getUid());

There are many properties set with the requestPayload and responsePaylod. Then finally we have

 log.debug(obj.toString());

There is this part of the Filter which converts the messages into JSON. What I ultimately want is this message in my logs(which are converted into JSON format using logstash-logback-encoder) but I dont want to deserialize or anything more to the above logic as this filter is applied largerly in our project,

I short how do we handle the messages which are already preconverted into JSON inside the logtash-logbackEncoder

If you know that all of the messages logged to a given logger are already pre-converted to JSON, then you could write your own JsonProvider that looked at the logger name from the LoggingEvent, and then pass the LoggingEvent's message to one of the generator.writeRaw* methods.

Not all the messages are pre-converted to JSON but the request which got to the filter JSONLogFilter have preconverted messages and as JSONFilter below extends OncePerRequestFilter, to every request to the server this filter is applied and hence message would be pre-converted to JSON. How do I specifically provide the generator.writeRaw methods to this only Filter or there any other way around

        <logger name="com.json.logging.config.log.JSONLogFilter" level="DEBUG" > <appender-ref 
         ref="GELF" /> </logger>

I can see some messages as below in the logs which is not in JSON format

        {
           "@timestamp": "2018-02-03T21:37:09.351+04:00",
           "@version": 1,
            "message": "SQL Error: 1146, SQLState: 42S02",
            "logger_name": "org.hibernate.engine.jdbc.spi.SqlExceptionHelper",
            "thread_name": "http-nio-3080-exec-9",
            "level": "WARN",
            "level_value": 30000,
            "HOSTNAME": "algo-11"
          }

In your custom JsonProvider, you can look at the LoggingEvent's logger name.

If the logging event's logger name is com.json.logging.config.log.JSONLogFilter, then assume that the message has already been pre-converted to json. Then you can call one of the generator.writeRaw methods to add the raw pre-converted json string to the logged json output.

The example you mentioned in your last comment has a different logger name (org.hibernate.engine.jdbc.spi.SqlExceptionHelper), so you wouldn't call generator.writeRaw for that event.

I tried in the below way and I was getting the message as raw JSON. But I am able to see the same message two times as below

  ` "{@timestamp":"2018-02-04T18:19:58.249+04:00",
    "@version":1,
    "message:"{\"type\":\"API_REQUEST\",\"uid\":\"ae09-345b-4303-b75736b7-c727d2b2d060\",\"sessionId\":null,\"remoteAddress\":\"0:0:0:0:0:0:0:1\",\"methodType\":\"POST\",\"headers\":\"{authorization={\\\"userId\\\":\\\"kld123445\\\",\\\"userKey\\\":\\\"wgfy4roy47o459849r4wr95455553\\\"}, content-length=76, referer=http://localhost:3080/swagger-ui.html, accept-language=en-US,en;q=0.9,ar;q=0.8, origin=http://localhost:3080, host=localhost:3080, connection=keep-alive, content-type=application/json, accept-encoding=gzip, deflate, br, accept=*/*, user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36}\",\"uri\":\"/user/priviledge/request/promotion\",\"query\":null,\"contentType\":\"application/json\",\"requestTime\":\"2018-02-04 18:19:58.046\",\"responseTime\":\"2018-02-04 18:19:58.246\",\"executionTimeMillisecond\":200,\"userId\":\"12\",\"requestPayload\":\"\",\r"esponsePayload\":\"{\\\"status\\\":\\\"FAIL\\\",\\\"statusCode\\\":1000,\\\"message\\\":\\\"User is not authorized to access this content.\\\",\\\"desc\\\":\\\"FAIL\\\",\\\"debug\\\":null,\\\"exception\\\":null,\\\"data\\\":null}\"}","logger_name":"com.kryptolabs.speakerswire.config.log.HttpLoggingFilter","thread_name":"http-nio-3080-exec-1","level":"DEBUG","level_value":10000,"HOSTNAME":"algo-11",
  "message":{"type":"API_REQUEST","uid":"ae09-345b-4303-b75736b7-
  c727d2b2d060","sessionId":null,"remoteAddress":"0:0:0:0:0:0:0:1","methodType":"POST","headers":"
  {authorization={\"userId\":\"kld123445\",\"userKey\":\"wgfy4roy47o459849r4wr95455553\"}, content-
 length=76, referer=http://localhost:3080/swagger-ui.html, accept-language=en-US,en;q=0.9,ar;q=0.8, origin=http://localhost:3080, host=localhost:3080, connection=keep-alive, content-type=application/json, accept-encoding=gzip, deflate, br, accept=*/*, user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36}","uri":"/user/priviledge/request/promotion","query":null,"contentType":"application/json","requestTime":"2018-02-04 18:19:58.046","responseTime":"2018-02-04 18:19:58.246","executionTimeMillisecond":200,"userId":"12","requestPayload":"","responsePayload":"{\"status\":\"FAIL\",\"statusCode\":1000,\"message\":\"User is not authorized to access this content.\",\"desc\":\"FAIL\",\"debug\":null,\"exception\":null,\"data\":null}"}

 }`

Below is my Java code

    `public static final String FIELD_MESSAGE = "message";
    public CustomMessageJsonProvider() {
        setFieldName(FIELD_MESSAGE);
    }
@Override
public void writeTo(JsonGenerator generator, ILoggingEvent event) throws IOException {
    if(event.getLoggerName().equals("com.json.logging.config.log.JSONLogFilter") && 
                 event.getMessage().contains("API_REQUEST")){
           generator.writeRawValue(event.getMessage());    
    }
    JsonWritingUtils.writeStringField(generator, getFieldName(), event.getFormattedMessage());
}
@Override
public void setFieldNames(LogstashFieldNames fieldNames) {
    setFieldName(fieldNames.getMessage());
}`

Here is the configuration inside logback.xml

        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
        <provider  class=com.json.logging.config.log..CustomMessageJsonProvider">
        </provider>
        <jsonFactoryDecorator 
                    class="com.json.logging.config.log..CustomEscapeLogstashDecorator"/>
       </encoder>

You'll need to disable the other json provider that provides the standard message field (the one that is getting escaped).

<fieldNames>
    <message>[ignore]</message>
</fieldNames>

Getting the messages, but the message field is at the last of the JSON, I like it to be after @version and how should I eliminate the \n and \t inside the responsePayload

{"@timestamp":"2018-01-31T19:21:27.624+04:00","@version":1,"message":"{\"type\":\"API_REQUEST\",\"uid\":\"35f53fe5-7c63-4936-a617-14601073a5f7\",\"sessionId\":\"9E998B68C5A6D66C30D215DE0B4B5EF3\",\"remoteAddress\":\"0:0:0:0:0:0:0:1\",\"methodType\":\"POST\",\"headers\":\"{content-length=76, accept-language=en-US,en;q=0.9, cookie=JSESSIONID=50D17B32E96A6A8850F2A5FD52A4C07D, postman-token=a5be9806-a0a6-3905-e45d-2bea9036f3ed, origin=chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop, accept=/, host=localhost:3080, connection=keep-alive, content-type=application/json, cache-control=no-cache, accept-encoding=gzip, deflate, br, user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36}\",\"uri\":\"/user/login\",\"query\":null,\"contentType\":\"application/json\",\"requestTime\":\"2018-01-31 19:21:26.024\",\"responseTime\":\"2018-01-31 19:21:27.604\",\"executionTimeMillisecond\":1580,\"requestPayload\":\"{\n \"\": \"+971\",\n \"phone\": \"545901009\",\n \"employeeType\": \"PHONE\"\n }\",\"responsePayload\":\"{\"status\":\"FAIL\",\"statusCode\":1000,\"message\":\"Internal Server Error\",\"desc\":\"FAIL\",\"debug\":\"org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.IllegalArgumentException: non null key required\n\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:982)\n\tat org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:648)\n\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:729)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)\n\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)\n\tat

To eliminate the escaped chars inside the requestPayload and responsePayload, you will have to modify your com.json.logging.config.log.JSONLogFilter to not escape them, since those are getting escaped by it. (i.e. it's not getting escaped by the LogstashEncoder).

If you want to change the ordering of the fields, you'll have to stop using LogstashEncoder, and instead use the LoggingEventCompositeJsonEncoder with the providers added in the order in which you want them.

Hi now I would like to add a new fieldName as SuccessMessage such that it should be shown in Kibana. This SuccessMessage field consists of the data from the successResponse object.

 log.debug("Success Message is {} , Failure Message is {}", successResponse, failureResponse)

How do I create a new field which contains the content of successResponse(it is an object of SuccessResponse class with fields name, statuscode, status) in JSON format and see it on kibana

Use StructuredArguments as mentioned here

Hi Phil,

I am getting enough out of it. Should we make changes where log.info or log.debug available inside the application source code.

Can you please provide me an example how to do basing on the log.debug statement which is inside application

 log.debug("Success Message is {} , Failure Message is {}",
                                                                           successResponse, failureResponse)

On Wed, Feb 14, 2018 at 1:01 AM, Phil Clay notifications@github.com wrote:

Use StructuredArguments as mentioned here https://github.com/logstash/logstash-logback-encoder#event-specific-custom-fields

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/logstash/logstash-logback-encoder/issues/264#issuecomment-365402879, or mute the thread https://github.com/notifications/unsubscribe-auth/AhkveiZPYjHJOYOnyANdL2NgcIQaioefks5tUfhEgaJpZM4R0L1k .

@philsttr : I have seen that the objects inside the employee object which contains various fields with values don't convert to JSON

    log.debug("This is employee Object : {}" ,  employee)

They are displayed as

"message":"This is employee Object: Employee(employeeId=16, city=ABCC, country=XYZZ ,countryCode=+921, email=null, phone=12222344, aboutMe=test),

Inside the below JSON

{"@timestamp":"2018-02-11T11:33:08.746+04:00","@version":1,"message":"This is employee Object: Employee(employeeId=16, city=Galazy, country=Pandora Ghats,countryCode=+91, email=null, phone=12222344, aboutMe=test),"thread_name":"Background-8","level":"DEBUG","level_value":10000,"HOSTNAME":"algo-11"}

I am in the assumption that logstash-logback-encoder converts the objects passed also in JSON. How do I make changes such that all the objects passed in the application to log.debug or log.info get converted to JSON format.

The message argument passed to the log methods will always end up in the message field of the json output. This is exactly what is happening in your example. Logstash itself is substituting the toString() representation of the employee argument into the message.

If you want to include additional fields in the json output, then you need to use structured arguments as mentioned here

For example...

log.debug("Some message", keyValue("employee", employee))

which would result in json output like this

{
    ...
    "message" : "Some message",
    "employee" : {
        ...
    }
}

@philsttr :

As with your comment Logstash itself is substituting the toString() representation of the employee argument into the message.

I think its not because of Logstash but its format is the toString representation of using Lombok plugin in our project. I just want these objects in JSON format and I am not able to figureout how to do using logstash logback-encoder and with the below example I have make changes all over my application and it is not working either

I tried in this way but it does not work

         `Employee employee=new Employee();
        employee.setFirstName("Sawyer");
        employee.setLastName("Ford");
        employee.setPhoneNumber("12334433");
        employee.setAddress("deddwde");
        employee.setDescription("HireriepriwerierwerHellorwero"); `

LOGGER.debug("Employee Object::::::{}", value("employeeString", employee) This above statement returns Employee Object::::::com.jsonlogging.demo.springboot.model.Employee@16c0507b

LOGGER.debug("Employee Object::::::{}", kv("employeeString", employee)); This is returning the value as Employee Object::::::employee= com.jsonlogging.demo.springboot.model.Employee@16c0507b

I am not getting pure JSON form the use of structured Arguments. Moreover in my applications there are thousands of log.debug statements with objects, so using structured arguments do we have to change all the debug statements throughout the application or can there be a centralized approach for this

I think its not because of Logstash but its format is the toString representation of using Lombok plugin in our project

You are correct here. I meant to say logback, not logstash in my statement. Logback will substitute the toString() representation of the arguments into the appropriate placeholder ({}) in the message field of a log event. This is just standard logback behavior.

I also think I see source of the confusion. StructuredArguments will only be included in the JSON output if using composite encoders/layouts with the arguments provider, or if you add the ArgumentsJsonProvider provider to your LogstashEncoder configuration like this

<encoder class="net.logstash.logback.encoder.LogstashEncoder">
    <provider class="net.logstash.logback.composite.loggingevent.ArgumentsJsonProvider"/>
</encoder>

With the above configuration, the following log statements...

        LOGGER.debug("1 Employee Object::::::{}", value("employeeString", employee));

        LOGGER.debug("2 Employee Object::::::{}", keyValue("employeeString", employee));

        LOGGER.debug("3 Employee Object::::::", value("employeeString", employee));

        LOGGER.debug("4 Employee Object::::::{}", employee);

will result in the following output (formatted for clarity):

{
  "@timestamp" : "2018-02-15T21:13:47.720-08:00",
  "@version" : 1,
  "message" : "1 Employee Object::::::com.jsonlogging.demo.springboot.model.Employee@3f72aff0",
  "logger_name" : "Main",
  "thread_name" : "main",
  "level" : "DEBUG",
  "level_value" : 10000,
  "employeeString" : {
    "firstName" : "Sawyer",
    "lastName" : "Ford",
    "phoneNumber" : "12334433",
    "address" : "deddwde",
    "description" : "HireriepriwerierwerHellorwero"
  }
}
{
  "@timestamp" : "2018-02-15T21:13:47.747-08:00",
  "@version" : 1,
  "message" : "2 Employee Object::::::employeeString=com.jsonlogging.demo.springboot.model.Employee@3f72aff0",
  "logger_name" : "Main",
  "thread_name" : "main",
  "level" : "DEBUG",
  "level_value" : 10000,
  "employeeString" : {
    "firstName" : "Sawyer",
    "lastName" : "Ford",
    "phoneNumber" : "12334433",
    "address" : "deddwde",
    "description" : "HireriepriwerierwerHellorwero"
  }
}
{
  "@timestamp" : "2018-02-15T21:13:47.747-08:00",
  "@version" : 1,
  "message" : "3 Employee Object::::::",
  "logger_name" : "Main",
  "thread_name" : "main",
  "level" : "DEBUG",
  "level_value" : 10000,
  "employeeString" : {
    "firstName" : "Sawyer",
    "lastName" : "Ford",
    "phoneNumber" : "12334433",
    "address" : "deddwde",
    "description" : "HireriepriwerierwerHellorwero"
  }
}
{
  "@timestamp" : "2018-02-15T21:13:47.747-08:00",
  "@version" : 1,
  "message" : "4 Employee Object::::::com.jsonlogging.demo.springboot.model.Employee@3f72aff0",
  "logger_name" : "Main",
  "thread_name" : "main",
  "level" : "DEBUG",
  "level_value" : 10000
}

Notice that the final log event does not include the argument, since it is not a structured argument. If you want to include non-structured arguments, you can do so like this:

<encoder class="net.logstash.logback.encoder.LogstashEncoder">
    <provider class="net.logstash.logback.composite.loggingevent.ArgumentsJsonProvider">
        <includeNonStructuredArguments>true</includeNonStructuredArguments>
    </provider>
</encoder>

In which case the final log event from above would be logged as:

{
  "@timestamp" : "2018-02-15T21:14:35.748-08:00",
  "@version" : 1,
  "message" : "4 Employee Object::::::com.jsonlogging.demo.springboot.model.Employee@25c17335",
  "logger_name" : "Main",
  "thread_name" : "main",
  "level" : "DEBUG",
  "level_value" : 10000,
  "arg0" : {
    "firstName" : "Sawyer",
    "lastName" : "Ford",
    "phoneNumber" : "12334433",
    "address" : "deddwde",
    "description" : "HireriepriwerierwerHellorwero"
  }
}

Note #169 will enable the structured args by default

@philsttr

Thanks, I got the formatted output But when I have a toString represntation of the object in my project as below, I get double values. How do I get the output such that I don't have the toString representation in the message field and only in the arg0

     @Override
     public String toString() {
              return "Employee{" +
                    "firstName='" + firstName + '\'' +
                   ", lastName='" + lastName + '\'' +
                   ", address='" + address + '\'' +
                   ", phoneNumber='" + phoneNumber + '\'' +
                   ", description='" + description + '\'' +
              '}';
       }

I get the output as

{
"@timestamp": "2018-02-16T15:03:23.923+04:00",
"@version": 1,
"message": "Normal Employee Object::::::Employee{firstName='Sawyer', lastName='Ford', address='qwweew', phoneNumber='123456778', description='Hireriepriwer'}",
"logger_name": "com.jsonlogging.demo.springboot.controller.EntryController",
"thread_name": "http-nio-8080-exec-1",
"level": "DEBUG",
"level_value": 10000,
"HOSTNAME": "algo-11",
"arg0": {
    "firstName": "Sawyer",
    "lastName": "Ford",
    "address": "qwweew",
    "phoneNumber": "123456778",
    "description": "Hireriepriwer"
}
}

AND

Is there any way I get the name of the arg0 in the JSON output as class name or name of the object to which the values of the arg0 to which it corresponds to without adding strucutured arguements of keyvalue/ kv in the application code( I have to make lot of changes then) for example

"arg0" : {
   "firstName" : "Sawyer",
   "lastName" : "Ford",
   "phoneNumber" : "12334433",
   "address" : "deddwde",
   "description" : "HireriepriwerierwerHellorwero"
 }

"Employee" : {
 "firstName" : "Sawyer",
 "lastName" : "Ford",
 "phoneNumber" : "12334433",
 "address" : "deddwde",
 "description" : "HireriepriwerierwerHellorwero"
   }

 "Organization" : {
"firstName" : "Sawyer",
"lastName" : "Ford",
"phoneNumber" : "12334433",
"address" : "deddwde",
"description" : "HireriepriwerierwerHellorwero"
  }

How do I get the output such that I don't have the toString representation in the message field and only in the arg0

You have two options to do this:

Don't include a placeholder ({}) for the argument in your message. Therefore, logback won't insert the argument.toString() into the message OR
Turn off the default MessageJsonProvider (as mentioned here), and add a new provider similar to MessageJsonProvider that gets the unformatted message from the loggingEvent. i.e. use loggingEvent.getMessage() instead of loggingEvent.getFormattedMessage()

For example:

public class UnformattedMessageJsonProvider extends MessageJsonProvider {

    @Override
    public void writeTo(JsonGenerator generator, ILoggingEvent event) throws IOException {
        JsonWritingUtils.writeStringField(generator, getFieldName(), event.getMessage());
    }
}

Is there any way I get the name of the arg0 in the JSON output as class name or name of the object to which the values of the arg0 to which it corresponds to without adding strucutured arguements of keyvalue/ kv in the application code

You would have to write your own json provider to do this. For example, something like this

public class NamedArgumentJsonProvider extends AbstractJsonProvider<LoggingEvent> {

    @Override
    public void writeTo(JsonGenerator generator, LoggingEvent event) throws IOException {
        for (Object argument : event.getArgumentArray()) {
            if (argument != null) {
                generator.writeFieldName(argument.getClass().getSimpleName());
                generator.writeObject(argument);
            }
        }
    }
}

@philsttr Is the above change configuration of UnformattedMessageJsonProvider and NamedArgumentJsonProvider is fine because I am getting the output string as below

    {
    "@timestamp": "2018-02-17T05:54:38.017+04:00",
    "@version": 1,
    "message": "\n\n\n=========================\nAUTO-CONFIGURATION REPORT\n=========================\n\n\nPositive matches:\n-----------------\n\n   DispatcherServletAutoConfiguration matched:\n      - @ConditionalOnClass found required class 'org.springframework.web.servlet.DispatcherServlet'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnWebApplication (required) found StandardServletEnvironment (OnWebApplicationCondition)\n\n   DispatcherServletAutoConfiguration.DispatcherServletConfiguration matched:\n      - @ConditionalOnClass found required class 'javax.servlet.ServletRegistration'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - Default DispatcherServlet did not find dispatcher servlet beans (DispatcherServletAutoConfiguration.DefaultDispatcherServletCondition)\n\n   DispatcherServletAutoConfiguration.DispatcherServletRegistrationConfiguration matched:\n      - @ConditionalOnClass found required class 'javax.servlet.ServletRegistration'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - DispatcherServlet Registration did not find servlet registration bean (DispatcherServletAutoConfiguration.DispatcherServletRegistrationCondition)\n\n   DispatcherServletAutoConfiguration.DispatcherServletRegistrationConfiguration#dispatcherServletRegistration matched:\n      - @ConditionalOnBean (names: dispatcherServlet; types: org.springframework.web.servlet.DispatcherServlet; SearchStrategy: all) found beans 'dispatcherServlet', 'dispatcherServlet' (OnBeanCondition)\n\n   EmbeddedServletContainerAutoConfiguration matched:\n      - @ConditionalOnWebApplication (required) found StandardServletEnvironment (OnWebApplicationCondition)\n\n   EmbeddedServletContainerAutoConfiguration.EmbeddedTomcat matched:\n      - @ConditionalOnClass found required classes 'javax.servlet.Servlet', 'org.apache.catalina.startup.Tomcat'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnMissingBean (types: org.springframework.boot.context.embedded.EmbeddedServletContainerFactory; SearchStrategy: current) did not find any beans (OnBeanCondition)\n\n   ErrorMvcAutoConfiguration matched:\n      - @ConditionalOnClass found required classes 'javax.servlet.Servlet', 'org.springframework.web.servlet.DispatcherServlet'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnWebApplication (required) found StandardServletEnvironment (OnWebApplicationCondition)\n\n   ErrorMvcAutoConfiguration#basicErrorController matched:\n      - @ConditionalOnMissingBean (types: org.springframework.boot.autoconfigure.web.ErrorController; SearchStrategy: current) did not find any beans (OnBeanCondition)\n\n   ErrorMvcAutoConfiguration#errorAttributes matched:\n      - @ConditionalOnMissingBean (types: org.springframework.boot.autoconfigure.web.ErrorAttributes; SearchStrategy: current) did not find any beans (OnBeanCondition)\n\n   ErrorMvcAutoConfiguration.DefaultErrorViewResolverConfiguration#conventionErrorViewResolver matched:\n      - @ConditionalOnBean (types: org.springframework.web.servlet.DispatcherServlet; SearchStrategy: all) found bean 'dispatcherServlet'; @ConditionalOnMissingBean (types: org.springframework.boot.autoconfigure.web.DefaultErrorViewResolver; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   ErrorMvcAutoConfiguration.WhitelabelErrorViewConfiguration matched:\n      - @ConditionalOnProperty (server.error.whitelabel.enabled) matched (OnPropertyCondition)\n      - ErrorTemplate Missing did not find error template view (ErrorMvcAutoConfiguration.ErrorTemplateMissingCondition)\n\n   ErrorMvcAutoConfiguration.WhitelabelErrorViewConfiguration#beanNameViewResolver matched:\n      - @ConditionalOnMissingBean (types: org.springframework.web.servlet.view.BeanNameViewResolver; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   ErrorMvcAutoConfiguration.WhitelabelErrorViewConfiguration#defaultErrorView matched:\n      - @ConditionalOnMissingBean (names: error; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   GenericCacheConfiguration matched:\n      - Cache org.springframework.boot.autoconfigure.cache.GenericCacheConfiguration automatic cache type (CacheCondition)\n\n   HttpEncodingAutoConfiguration matched:\n      - @ConditionalOnClass found required class 'org.springframework.web.filter.CharacterEncodingFilter'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnWebApplication (required) found StandardServletEnvironment (OnWebApplicationCondition)\n      - @ConditionalOnProperty (spring.http.encoding.enabled) matched (OnPropertyCondition)\n\n   HttpEncodingAutoConfiguration#characterEncodingFilter matched:\n      - @ConditionalOnMissingBean (types: org.springframework.web.filter.CharacterEncodingFilter; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   HttpMessageConvertersAutoConfiguration matched:\n      - @ConditionalOnClass found required class 'org.springframework.http.converter.HttpMessageConverter'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   HttpMessageConvertersAutoConfiguration#messageConverters matched:\n      - @ConditionalOnMissingBean (types: org.springframework.boot.autoconfigure.web.HttpMessageConverters; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   HttpMessageConvertersAutoConfiguration.StringHttpMessageConverterConfiguration matched:\n      - @ConditionalOnClass found required class 'org.springframework.http.converter.StringHttpMessageConverter'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   HttpMessageConvertersAutoConfiguration.StringHttpMessageConverterConfiguration#stringHttpMessageConverter matched:\n      - @ConditionalOnMissingBean (types: org.springframework.http.converter.StringHttpMessageConverter; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   JacksonAutoConfiguration matched:\n      - @ConditionalOnClass found required class 'com.fasterxml.jackson.databind.ObjectMapper'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   JacksonAutoConfiguration.Jackson2ObjectMapperBuilderCustomizerConfiguration matched:\n      - @ConditionalOnClass found required classes 'com.fasterxml.jackson.databind.ObjectMapper', 'org.springframework.http.converter.json.Jackson2ObjectMapperBuilder'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   JacksonAutoConfiguration.JacksonObjectMapperBuilderConfiguration matched:\n      - @ConditionalOnClass found required classes 'com.fasterxml.jackson.databind.ObjectMapper', 'org.springframework.http.converter.json.Jackson2ObjectMapperBuilder'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   JacksonAutoConfiguration.JacksonObjectMapperBuilderConfiguration#jacksonObjectMapperBuilder matched:\n      - @ConditionalOnMissingBean (types: org.springframework.http.converter.json.Jackson2ObjectMapperBuilder; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   JacksonAutoConfiguration.JacksonObjectMapperConfiguration matched:\n      - @ConditionalOnClass found required classes 'com.fasterxml.jackson.databind.ObjectMapper', 'org.springframework.http.converter.json.Jackson2ObjectMapperBuilder'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   JacksonAutoConfiguration.JacksonObjectMapperConfiguration#jacksonObjectMapper matched:\n      - @ConditionalOnMissingBean (types: com.fasterxml.jackson.databind.ObjectMapper; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   JacksonHttpMessageConvertersConfiguration.MappingJackson2HttpMessageConverterConfiguration matched:\n      - @ConditionalOnClass found required class 'com.fasterxml.jackson.databind.ObjectMapper'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnProperty (spring.http.converters.preferred-json-mapper=jackson) matched (OnPropertyCondition)\n      - @ConditionalOnBean (types: com.fasterxml.jackson.databind.ObjectMapper; SearchStrategy: all) found bean 'jacksonObjectMapper' (OnBeanCondition)\n\n   JacksonHttpMessageConvertersConfiguration.MappingJackson2HttpMessageConverterConfiguration#mappingJackson2HttpMessageConverter matched:\n      - @ConditionalOnMissingBean (types: org.springframework.http.converter.json.MappingJackson2HttpMessageConverter; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   JmxAutoConfiguration matched:\n      - @ConditionalOnClass found required class 'org.springframework.jmx.export.MBeanExporter'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnProperty (spring.jmx.enabled=true) matched (OnPropertyCondition)\n\n   JmxAutoConfiguration#mbeanExporter matched:\n      - @ConditionalOnMissingBean (types: org.springframework.jmx.export.MBeanExporter; SearchStrategy: current) did not find any beans (OnBeanCondition)\n\n   JmxAutoConfiguration#mbeanServer matched:\n      - @ConditionalOnMissingBean (types: javax.management.MBeanServer; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   JmxAutoConfiguration#objectNamingStrategy matched:\n      - @ConditionalOnMissingBean (types: org.springframework.jmx.export.naming.ObjectNamingStrategy; SearchStrategy: current) did not find any beans (OnBeanCondition)\n\n   MultipartAutoConfiguration matched:\n      - @ConditionalOnClass found required classes 'javax.servlet.Servlet', 'org.springframework.web.multipart.support.StandardServletMultipartResolver', 'javax.servlet.MultipartConfigElement'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnProperty (spring.http.multipart.enabled) matched (OnPropertyCondition)\n\n   MultipartAutoConfiguration#multipartConfigElement matched:\n      - @ConditionalOnMissingBean (types: javax.servlet.MultipartConfigElement; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   MultipartAutoConfiguration#multipartResolver matched:\n      - @ConditionalOnMissingBean (types: org.springframework.web.multipart.MultipartResolver; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   NoOpCacheConfiguration matched:\n      - Cache org.springframework.boot.autoconfigure.cache.NoOpCacheConfiguration automatic cache type (CacheCondition)\n\n   PropertyPlaceholderAutoConfiguration#propertySourcesPlaceholderConfigurer matched:\n      - @ConditionalOnMissingBean (types: org.springframework.context.support.PropertySourcesPlaceholderConfigurer; SearchStrategy: current) did not find any beans (OnBeanCondition)\n\n   RedisCacheConfiguration matched:\n      - Cache org.springframework.boot.autoconfigure.cache.RedisCacheConfiguration automatic cache type (CacheCondition)\n\n   ServerPropertiesAutoConfiguration matched:\n      - @ConditionalOnWebApplication (required) found StandardServletEnvironment (OnWebApplicationCondition)\n\n   ServerPropertiesAutoConfiguration#serverProperties matched:\n      - @ConditionalOnMissingBean (types: org.springframework.boot.autoconfigure.web.ServerProperties; SearchStrategy: current) did not find any beans (OnBeanCondition)\n\n   SimpleCacheConfiguration matched:\n      - Cache org.springframework.boot.autoconfigure.cache.SimpleCacheConfiguration automatic cache type (CacheCondition)\n\n   ValidationAutoConfiguration matched:\n      - @ConditionalOnClass found required class 'javax.validation.executable.ExecutableValidator'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnResource found location classpath:META-INF/services/javax.validation.spi.ValidationProvider (OnResourceCondition)\n\n   ValidationAutoConfiguration#defaultValidator matched:\n      - @ConditionalOnMissingBean (types: javax.validation.Validator; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   ValidationAutoConfiguration#methodValidationPostProcessor matched:\n      - @ConditionalOnMissingBean (types: org.springframework.validation.beanvalidation.MethodValidationPostProcessor; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   WebClientAutoConfiguration.RestTemplateConfiguration matched:\n      - @ConditionalOnClass found required class 'org.springframework.web.client.RestTemplate'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   WebClientAutoConfiguration.RestTemplateConfiguration#restTemplateBuilder matched:\n      - @ConditionalOnMissingBean (types: org.springframework.boot.web.client.RestTemplateBuilder; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   WebMvcAutoConfiguration matched:\n      - @ConditionalOnClass found required classes 'javax.servlet.Servlet', 'org.springframework.web.servlet.DispatcherServlet', 'org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnWebApplication (required) found StandardServletEnvironment (OnWebApplicationCondition)\n      - @ConditionalOnMissingBean (types: org.springframework.web.servlet.config.annotation.WebMvcConfigurationSupport; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   WebMvcAutoConfiguration#hiddenHttpMethodFilter matched:\n      - @ConditionalOnMissingBean (types: org.springframework.web.filter.HiddenHttpMethodFilter; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   WebMvcAutoConfiguration#httpPutFormContentFilter matched:\n      - @ConditionalOnProperty (spring.mvc.formcontent.putfilter.enabled) matched (OnPropertyCondition)\n      - @ConditionalOnMissingBean (types: org.springframework.web.filter.HttpPutFormContentFilter; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   WebMvcAutoConfiguration.WebMvcAutoConfigurationAdapter#defaultViewResolver matched:\n      - @ConditionalOnMissingBean (types: org.springframework.web.servlet.view.InternalResourceViewResolver; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   WebMvcAutoConfiguration.WebMvcAutoConfigurationAdapter#requestContextFilter matched:\n      - @ConditionalOnMissingBean (types: org.springframework.web.context.request.RequestContextListener,org.springframework.web.filter.RequestContextFilter; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   WebMvcAutoConfiguration.WebMvcAutoConfigurationAdapter#viewResolver matched:\n      - @ConditionalOnBean (types: org.springframework.web.servlet.ViewResolver; SearchStrategy: all) found beans 'defaultViewResolver', 'beanNameViewResolver', 'mvcViewResolver'; @ConditionalOnMissingBean (names: viewResolver; types: org.springframework.web.servlet.view.ContentNegotiatingViewResolver; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   WebMvcAutoConfiguration.WebMvcAutoConfigurationAdapter.FaviconConfiguration matched:\n      - @ConditionalOnProperty (spring.mvc.favicon.enabled) matched (OnPropertyCondition)\n\n   WebSocketAutoConfiguration matched:\n      - @ConditionalOnClass found required classes 'javax.servlet.Servlet', 'javax.websocket.server.ServerContainer'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n      - @ConditionalOnWebApplication (required) found StandardServletEnvironment (OnWebApplicationCondition)\n\n   WebSocketAutoConfiguration.TomcatWebSocketConfiguration matched:\n      - @ConditionalOnClass found required classes 'org.apache.catalina.startup.Tomcat', 'org.apache.tomcat.websocket.server.WsSci'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   WebSocketAutoConfiguration.TomcatWebSocketConfiguration#websocketContainerCustomizer matched:\n      - @ConditionalOnJava (1.7 or newer) found 1.8 (OnJavaCondition)\n      - @ConditionalOnMissingBean (names: websocketContainerCustomizer; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n\nNegative matches:\n-----------------\n\n   ActiveMQAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'javax.jms.ConnectionFactory', 'org.apache.activemq.ActiveMQConnectionFactory' (OnClassCondition)\n\n   AopAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.aspectj.lang.annotation.Aspect', 'org.aspectj.lang.reflect.Advice' (OnClassCondition)\n\n   ArtemisAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'javax.jms.ConnectionFactory', 'org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory' (OnClassCondition)\n\n   BatchAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.batch.core.launch.JobLauncher', 'org.springframework.jdbc.core.JdbcOperations' (OnClassCondition)\n\n   CacheAutoConfiguration:\n      Did not match:\n         - @ConditionalOnBean (types: org.springframework.cache.interceptor.CacheAspectSupport; SearchStrategy: all) did not find any beans (OnBeanCondition)\n      Matched:\n         - @ConditionalOnClass found required class 'org.springframework.cache.CacheManager'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)\n\n   CacheAutoConfiguration.CacheManagerJpaDependencyConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean' (OnClassCondition)\n         - Ancestor org.springframework.boot.autoconfigure.cache.CacheAutoConfiguration did not match (ConditionEvaluationReport.AncestorsMatchedCondition)\n\n   CaffeineCacheConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.github.benmanes.caffeine.cache.Caffeine', 'org.springframework.cache.caffeine.CaffeineCacheManager' (OnClassCondition)\n\n   CassandraAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.datastax.driver.core.Cluster' (OnClassCondition)\n\n   CassandraDataAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.datastax.driver.core.Cluster', 'org.springframework.data.cassandra.core.CassandraAdminOperations' (OnClassCondition)\n\n   CassandraRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.datastax.driver.core.Session', 'org.springframework.data.cassandra.repository.CassandraRepository' (OnClassCondition)\n\n   CloudAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.cloud.config.java.CloudScanConfiguration' (OnClassCondition)\n\n   CouchbaseAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.couchbase.client.java.CouchbaseBucket', 'com.couchbase.client.java.Cluster' (OnClassCondition)\n\n   CouchbaseCacheConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.couchbase.client.java.Bucket', 'com.couchbase.client.spring.cache.CouchbaseCacheManager' (OnClassCondition)\n\n   CouchbaseDataAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.couchbase.client.java.Bucket', 'org.springframework.data.couchbase.repository.CouchbaseRepository' (OnClassCondition)\n\n   CouchbaseRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.couchbase.client.java.Bucket', 'org.springframework.data.couchbase.repository.CouchbaseRepository' (OnClassCondition)\n\n   DataSourceAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType' (OnClassCondition)\n\n   DataSourceTransactionManagerAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.jdbc.core.JdbcTemplate', 'org.springframework.transaction.PlatformTransactionManager' (OnClassCondition)\n\n   DeviceDelegatingViewResolverAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.mobile.device.view.LiteDeviceDelegatingViewResolver' (OnClassCondition)\n\n   DeviceResolverAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.mobile.device.DeviceResolverHandlerInterceptor', 'org.springframework.mobile.device.DeviceHandlerMethodArgumentResolver' (OnClassCondition)\n\n   DispatcherServletAutoConfiguration.DispatcherServletConfiguration#multipartResolver:\n      Did not match:\n         - @ConditionalOnBean (types: org.springframework.web.multipart.MultipartResolver; SearchStrategy: all) did not find any beans (OnBeanCondition)\n\n   EhCacheCacheConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'net.sf.ehcache.Cache', 'org.springframework.cache.ehcache.EhCacheCacheManager' (OnClassCondition)\n\n   ElasticsearchAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.elasticsearch.client.Client', 'org.springframework.data.elasticsearch.client.TransportClientFactoryBean', 'org.springframework.data.elasticsearch.client.NodeClientFactoryBean' (OnClassCondition)\n\n   ElasticsearchDataAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.elasticsearch.client.Client', 'org.springframework.data.elasticsearch.core.ElasticsearchTemplate' (OnClassCondition)\n\n   ElasticsearchRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.elasticsearch.client.Client', 'org.springframework.data.elasticsearch.repository.ElasticsearchRepository' (OnClassCondition)\n\n   EmbeddedLdapAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.unboundid.ldap.listener.InMemoryDirectoryServer' (OnClassCondition)\n\n   EmbeddedMongoAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.mongodb.Mongo', 'de.flapdoodle.embed.mongo.MongodStarter' (OnClassCondition)\n\n   EmbeddedServletContainerAutoConfiguration.EmbeddedJetty:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.eclipse.jetty.server.Server', 'org.eclipse.jetty.util.Loader', 'org.eclipse.jetty.webapp.WebAppContext' (OnClassCondition)\n\n   EmbeddedServletContainerAutoConfiguration.EmbeddedUndertow:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'io.undertow.Undertow', 'org.xnio.SslClientAuthMode' (OnClassCondition)\n\n   FacebookAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.social.config.annotation.SocialConfigurerAdapter', 'org.springframework.social.facebook.connect.FacebookConnectionFactory' (OnClassCondition)\n\n   FallbackWebSecurityAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.security.config.annotation.web.configuration.EnableWebSecurity' (OnClassCondition)\n\n   FlywayAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.flywaydb.core.Flyway' (OnClassCondition)\n\n   FreeMarkerAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'freemarker.template.Configuration', 'org.springframework.ui.freemarker.FreeMarkerConfigurationFactory' (OnClassCondition)\n\n   GroovyTemplateAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'groovy.text.markup.MarkupTemplateEngine' (OnClassCondition)\n\n   GsonAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.google.gson.Gson' (OnClassCondition)\n\n   GsonHttpMessageConvertersConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.google.gson.Gson' (OnClassCondition)\n\n   GuavaCacheConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.google.common.cache.CacheBuilder', 'org.springframework.cache.guava.GuavaCacheManager' (OnClassCondition)\n\n   H2ConsoleAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.h2.server.web.WebServlet' (OnClassCondition)\n\n   HazelcastAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.hazelcast.core.HazelcastInstance' (OnClassCondition)\n\n   HazelcastCacheConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.hazelcast.core.HazelcastInstance', 'com.hazelcast.spring.cache.HazelcastCacheManager' (OnClassCondition)\n\n   HazelcastJpaDependencyAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.hazelcast.core.HazelcastInstance', 'org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean' (OnClassCondition)\n\n   HibernateJpaAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean', 'javax.persistence.EntityManager' (OnClassCondition)\n\n   HypermediaAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.hateoas.Resource', 'org.springframework.plugin.core.Plugin' (OnClassCondition)\n\n   InfinispanCacheConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.infinispan.spring.provider.SpringEmbeddedCacheManager' (OnClassCondition)\n\n   IntegrationAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.integration.config.EnableIntegration' (OnClassCondition)\n\n   JCacheCacheConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'javax.cache.Caching', 'org.springframework.cache.jcache.JCacheCacheManager' (OnClassCondition)\n\n   JacksonAutoConfiguration.JodaDateTimeJacksonConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.joda.time.DateTime', 'com.fasterxml.jackson.datatype.joda.ser.DateTimeSerializer', 'com.fasterxml.jackson.datatype.joda.cfg.JacksonJodaDateFormat' (OnClassCondition)\n\n   JacksonAutoConfiguration.ParameterNamesModuleConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.fasterxml.jackson.module.paramnames.ParameterNamesModule' (OnClassCondition)\n\n   JacksonHttpMessageConvertersConfiguration.MappingJackson2XmlHttpMessageConverterConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.fasterxml.jackson.dataformat.xml.XmlMapper' (OnClassCondition)\n\n   JdbcTemplateAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.jdbc.core.JdbcTemplate' (OnClassCondition)\n\n   JerseyAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.glassfish.jersey.server.spring.SpringComponentProvider' (OnClassCondition)\n\n   JestAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'io.searchbox.client.JestClient' (OnClassCondition)\n\n   JmsAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'javax.jms.Message', 'org.springframework.jms.core.JmsTemplate' (OnClassCondition)\n\n   JndiConnectionFactoryAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.jms.core.JmsTemplate' (OnClassCondition)\n\n   JndiDataSourceAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType' (OnClassCondition)\n\n   JooqAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.jooq.DSLContext' (OnClassCondition)\n\n   JpaRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.data.jpa.repository.JpaRepository' (OnClassCondition)\n\n   JtaAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'javax.transaction.Transaction' (OnClassCondition)\n\n   KafkaAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.kafka.core.KafkaTemplate' (OnClassCondition)\n\n   LdapAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.ldap.core.ContextSource' (OnClassCondition)\n\n   LdapDataAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.data.ldap.repository.LdapRepository' (OnClassCondition)\n\n   LdapRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.data.ldap.repository.LdapRepository' (OnClassCondition)\n\n   LinkedInAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.social.config.annotation.SocialConfigurerAdapter', 'org.springframework.social.linkedin.connect.LinkedInConnectionFactory' (OnClassCondition)\n\n   LiquibaseAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'liquibase.integration.spring.SpringLiquibase' (OnClassCondition)\n\n   MailSenderAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'javax.mail.internet.MimeMessage' (OnClassCondition)\n\n   MailSenderValidatorAutoConfiguration:\n      Did not match:\n         - @ConditionalOnProperty (spring.mail.test-connection) did not find property 'test-connection' (OnPropertyCondition)\n\n   MessageSourceAutoConfiguration:\n      Did not match:\n         - ResourceBundle did not find bundle with basename messages (MessageSourceAutoConfiguration.ResourceBundleCondition)\n\n   MongoAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.mongodb.MongoClient' (OnClassCondition)\n\n   MongoDataAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.mongodb.Mongo', 'org.springframework.data.mongodb.core.MongoTemplate' (OnClassCondition)\n\n   MongoRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'com.mongodb.Mongo', 'org.springframework.data.mongodb.repository.MongoRepository' (OnClassCondition)\n\n   MustacheAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.samskivert.mustache.Mustache' (OnClassCondition)\n\n   Neo4jDataAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.neo4j.ogm.session.SessionFactory', 'org.springframework.transaction.PlatformTransactionManager' (OnClassCondition)\n\n   Neo4jRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.neo4j.ogm.session.Neo4jSession', 'org.springframework.data.neo4j.repository.GraphRepository' (OnClassCondition)\n\n   OAuth2AutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.security.oauth2.common.OAuth2AccessToken' (OnClassCondition)\n\n   PersistenceExceptionTranslationAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.dao.annotation.PersistenceExceptionTranslationPostProcessor' (OnClassCondition)\n\n   ProjectInfoAutoConfiguration#buildProperties:\n      Did not match:\n         - @ConditionalOnResource did not find resource '${spring.info.build.location:classpath:META-INF/build-info.properties}' (OnResourceCondition)\n\n   ProjectInfoAutoConfiguration#gitProperties:\n      Did not match:\n         - GitResource did not find git info at classpath:git.properties (ProjectInfoAutoConfiguration.GitResourceAvailableCondition)\n\n   RabbitAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.amqp.rabbit.core.RabbitTemplate', 'com.rabbitmq.client.Channel' (OnClassCondition)\n\n   ReactorAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'reactor.spring.context.config.EnableReactor', 'reactor.Environment' (OnClassCondition)\n\n   RedisAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.data.redis.connection.jedis.JedisConnection', 'org.springframework.data.redis.core.RedisOperations', 'redis.clients.jedis.Jedis' (OnClassCondition)\n\n   RedisRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'redis.clients.jedis.Jedis', 'org.springframework.data.redis.repository.configuration.EnableRedisRepositories' (OnClassCondition)\n\n   RepositoryRestMvcAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.data.rest.webmvc.config.RepositoryRestMvcConfiguration' (OnClassCondition)\n\n   SecurityAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.security.authentication.AuthenticationManager', 'org.springframework.security.config.annotation.authentication.configurers.GlobalAuthenticationConfigurerAdapter' (OnClassCondition)\n\n   SecurityFilterAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.security.web.context.AbstractSecurityWebApplicationInitializer', 'org.springframework.security.config.http.SessionCreationPolicy' (OnClassCondition)\n\n   SendGridAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'com.sendgrid.SendGrid' (OnClassCondition)\n\n   SessionAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.session.Session' (OnClassCondition)\n\n   SitePreferenceAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.mobile.device.site.SitePreferenceHandlerInterceptor', 'org.springframework.mobile.device.site.SitePreferenceHandlerMethodArgumentResolver' (OnClassCondition)\n\n   SocialWebAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.social.connect.web.ConnectController', 'org.springframework.social.config.annotation.SocialConfigurerAdapter' (OnClassCondition)\n\n   SolrAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.apache.solr.client.solrj.impl.HttpSolrClient', 'org.apache.solr.client.solrj.impl.CloudSolrClient' (OnClassCondition)\n\n   SolrRepositoriesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.apache.solr.client.solrj.SolrClient', 'org.springframework.data.solr.repository.SolrRepository' (OnClassCondition)\n\n   SpringApplicationAdminJmxAutoConfiguration:\n      Did not match:\n         - @ConditionalOnProperty (spring.application.admin.enabled=true) did not find property 'enabled' (OnPropertyCondition)\n\n   SpringDataWebAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.data.web.PageableHandlerMethodArgumentResolver' (OnClassCondition)\n\n   ThymeleafAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.thymeleaf.spring4.SpringTemplateEngine' (OnClassCondition)\n\n   TransactionAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.transaction.PlatformTransactionManager' (OnClassCondition)\n\n   TwitterAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required classes 'org.springframework.social.config.annotation.SocialConfigurerAdapter', 'org.springframework.social.twitter.connect.TwitterConnectionFactory' (OnClassCondition)\n\n   WebMvcAutoConfiguration.ResourceChainCustomizerConfiguration:\n      Did not match:\n         - @ConditionalOnEnabledResourceChain did not find class org.webjars.WebJarAssetLocator (OnEnabledResourceChainCondition)\n\n   WebMvcAutoConfiguration.WebMvcAutoConfigurationAdapter#beanNameViewResolver:\n      Did not match:\n         - @ConditionalOnMissingBean (types: org.springframework.web.servlet.view.BeanNameViewResolver; SearchStrategy: all) found bean 'beanNameViewResolver' (OnBeanCondition)\n\n   WebMvcAutoConfiguration.WebMvcAutoConfigurationAdapter#dateFormatter:\n      Did not match:\n         - @ConditionalOnProperty (spring.mvc.date-format) did not find property 'date-format' (OnPropertyCondition)\n\n   WebMvcAutoConfiguration.WebMvcAutoConfigurationAdapter#localeResolver:\n      Did not match:\n         - @ConditionalOnProperty (spring.mvc.locale) did not find property 'locale' (OnPropertyCondition)\n\n   WebServicesAutoConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.springframework.ws.transport.http.MessageDispatcherServlet' (OnClassCondition)\n\n   WebSocketAutoConfiguration.JettyWebSocketConfiguration:\n      Did not match:\n         - @ConditionalOnClass did not find required class 'org.eclipse.jetty.websocket.jsr356.server.deploy.WebSocketServerContainerInitializer' (OnClassCondition)\n\n   WebSocketAutoConfiguration.UndertowWebSocketConfiguration:{"
    @timestamp ":"
    2018 - 02 - 17 T05: 54: 49.825 + 04: 00 ","
    @version ":1,"
    message ":"
    Normal Employee Object::::::{}
    ","
    Employee ":{"
    firstName ":"
    Sawyer ","
    lastName ":"
    Fored ","
    address ":"
    SWWEWEWE ","
    phoneNumber ":"
    123443434 ","
    description ":"
    Hello World!!"}}

Below is my configuration

      <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
            <timestamp/>
            <version/>
            <!-- https://github.com/logstash/logstash-logback-encoder#provider_pattern -->
            <pattern>
                <pattern>
                    {
                    "logMessage": "%message"
                    }
                </pattern>
            </pattern>
            <!-- log guid support -->
            <mdc/>
            <!-- StructuredArgument and Marker support -->
            <arguments/>
            <logstashMarkers/>

                <provider class="net.logstash.logback.composite.loggingevent.ArgumentsJsonProvider">
                    <includeNonStructuredArguments>true</includeNonStructuredArguments>
                </provider>
                <provider class="config.UnformattedMessageJsonProvider">
                </provider>
                <provider class="config.NamedArgumentJsonProvider">
                </provider>
            </providers>
        </encoder>

A few comments...

1) You do not need to include

                <provider class="net.logstash.logback.composite.loggingevent.ArgumentsJsonProvider">
                    <includeNonStructuredArguments>true</includeNonStructuredArguments>
                </provider>

Because the <arguments/> element will add the arguments json provider.

You don't need either of them if you are not going to use StructuredArguments.

2) You're also including the formatted message with the pattern provider. I thought you did not want the formatted message.

3) The output you provided looks like it's from two different log events. Were these output via the console appender? If so, they might be from two different threads. If that is the case, then you can put the console appender behind an AsyncDisruptorAppender so that log messages don't write on top of each other

@philsttr :

My Whole config is this, I do not want the formatted output., but when I turn off the MessageProvider I dont get the message with field in the JSON output and if I add I finally get the output as available in the last

 `<appender name="jsonAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <File>${LOG_PATH}</File>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <provider class="config.UnformattedMessageJsonProvider"/>                 
            <provider class="config.NamedArgumentJsonProvider"/>
    </encoder>`

Even then i get the below output in logfile, and also in console

**1. What is the exact right configuration?

Which encoder I have to choose ?
What should be the configuration if LogstashEncoder is chosen?
What should be the configuration LoggingEventCompositeJsonEncoder is chosen?
I get the clubbed message even though I have tried with both of them. What things I am missing here?
What is the primary difference between LogstashEncoder and LoggingEventCompositeJsonEncoder. I want to make clear*

{ "@timestamp": "2018-02-17T06:46:13.695+04:00", "@version": 1, "message": "INFO", "logger_name": "com.jsonlogging.demo.springboot.controller.EntryController", "thread_name": "http-nio-8080-exec-1", "level": "INFO", "level_value": 20000, "HOSTNAME": "algo-11" { "@timestamp": "2018-02-17T06:46:13.696+04:00", "@version": 1, "message": "DEBUG", "logger_name": "com.jsonlogging.demo.springboot.controller.EntryController", "thread_name": "http-nio-8080-exec-1", "level": "DEBUG", "level_value": 10000, "HOSTNAME": "algo-11" { "@timestamp": "2018-02-17T06:46:13.696+04:00", "@version": 1, "message": "WARN", "logger_name": "com.jsonlogging.demo.springboot.controller.EntryController", "thread_name": "http-nio-8080-exec-1", "level": "WARN", "level_value": 30000, "HOSTNAME": "algo-11" { "@timestamp": "2018-02-17T06:46:13.696+04:00", "@version": 1, "message": "ERROR", "logger_name": "com.jsonlogging.demo.springboot.controller.EntryController", "thread_name": "http-nio-8080-exec-1", "level": "ERROR", "level_value": 40000, "HOSTNAME": "algo-11" { "@timestamp": "2018-02-17T06:46:13.737+04:00", "@version": 1, "message": "Normal Employee Object::::::Employee{firstName='Sawyer', lastName='Fored', address='SWWEWEWE', phoneNumber='123443434', description='Hello World!!'}", "logger_name": "com.jsonlogging.demo.springboot.controller.EntryController", "thread_name": "http-nio-8080-exec-1", "level": "DEBUG", "level_value": 10000, "HOSTNAME": "algo-11", "Employee": { "firstName": "Sawyer", "lastName": "Fored", "address": "SWWEWEWE", "phoneNumber": "123443434", "description": "Hello World!!" } }

LoggingEventCompositeJsonEncoder encodes LoggingEvents as JSON by using JsonProviders. By default, LoggingEventCompositeJsonEncoder has no providers. It's up to you to add the providers that you want to get the output you want.

Each provider contributes something to the JSON output by the encoder. For example, the timestamp provider will add the LoggingEvent's timestamp to the JSON output. logstash-logback-encoder comes with many providers, but you can also add your own if those provided by logstash-logback-encoder do not handle your use case.

LogstashEncoder is a subclass of LoggingEventCompositeJsonEncoder with a set of providers pre-configured as sensible defaults.

Both LogstashEncoder and LoggingEventCompositeJsonEncoder use providers to contribute to the output. It's just that LogstashEncoder starts with a set of default providers to handle common use cases, and LoggingEventCompositeJsonEncoder starts with no providers.

For simple use cases, users can just use LogstashEncoder.

For complex use cases, it might be easier to start with LoggingEventCompositeJsonEncoder, and add the exact providers what you want, rather than starting with LogstashEncoder, and trying to remove/add/override functionality. For example, if you want to customize the ordering of fields in the JSON output, you can configure LoggingEventCompositeJsonEncoder with providers in the order you want. Or if you want to override some of the providers used by LogstashEncoder (like your desire to not output the formatted message field), it might be easier to use LoggingEventCompositeJsonEncoder

In summary, for your use case, I would recommend LoggingEventCompositeJsonEncoder, but you could probably still use LogstashEncoder if you wanted.

Here's what your configuration could potentially look like using the LoggingEventCompositeJsonEncoder. You can also add any other provider you want, depending on what you want included in the JSON output.

(Notice that the rawMessage provider is used here. This can be used instead of your UnformattedMessageJsonProvider. I didn't realize that logstash-logback-encoder already had a provider that outputs the unformatted message.)

<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
    <providers>
        <!-- Write a @timestamp field containing the LoggingEvent's timestamp-->
        <timestamp/>

        <!-- Write a @version field containing a value of 1-->
        <version/>

        <!-- Write a message field containing the raw/unformatted LoggingEvent message-->
        <rawMessage>
            <fieldName>message</fieldName>
        </rawMessage>

        <!-- Write the LoggingEvent's arguments as separate fields-->
        <provider class="config.NamedArgumentJsonProvider"/>
    </providers>
</encoder>

Regarding the clubbed messages, there is probably an exception occurring while generating the JSON output. I would recommend registering a logback status listener so you can see error messages and exceptions from logback. You can use <configuration debug="true"> or <statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />

@philsttr :

when I give

              generator.writeObjectField(fieldName, argument);

It is not capturing null values for example, the Employee class is

      @JsonInclude(JsonInclude.Include.NON_NULL)

  Employee employee=new Employee();
        employee.setFirstName("Sawyer");
        employee.setLastName("Fored");
        employee.setPhoneNumber("123443434");
        employee.setAddress("SWWEWEWE");
        employee.setDescription(null);

I get as

  {"firstName":"Sawyer","lastName":"Fored","address":"SWWEWEWE","phoneNumber":"123443434"}

The description:null value is not captured. How do I get it

I have tried below but not able to get

                objectMapper =  objectMapper.setSerializationInclusion(JsonInclude.Include.ALWAYS);
                generator.setCodec(objectMapper);
              generator.writeObjectField(fieldName, argument);

I believe objectMapper.setSerializationInclusion sets the default property inclusion value, which is only used if a class is not annotated with @JsonInclude. Since your Employee class is annotated with @JsonInclude(JsonInclude.Include.NON_NULL), the value from the annotation on the Employee class will take precedence. And since it's using JsonInclude.Include.NON_NULL, only non-null fields will be serialized.

I don't think jackson provides a way to override the inclusion rules for all classes. But for specific classes (e.g. your Employee class), you could use a mix-in that specifies a different value for @JsonInclude. The @JsonInclude annotation on the mix-in class will override the @JsonInclude annotation on the Employee class.

I recommend asking jackson-specific questions on the jackson users mailing list.

As a side note... you should configure the ObjectMapper/JsonFactory in a JsonFactoryDecorator, rather than in a JsonProvider. See here. The reason configuring the ObjectMapper/JsonFactory should be done in a JsonFactoryDecorator is because once an ObjectMapper is used for the first time, some of its configuration values cannot be changed. The JsonFactoryDecorator will be invoked prior to the ObjectMapper/JsonFactory being used for the first time.

I believe that is the way to do it.

Can you provide the complete source of your decorator? And also your encoder xml configuration which enables your decorator?

@philsttr

I have got two questions now

I have overridden the ArgumentsProvider and what is use of below configuration of ArgumentsProvider, what is getFieldName() value. because I get the getFieldName() value as Null. My Application do not reach to the first if condition and else if condition.

 `  if (argument instanceof StructuredArgument) {
             if (!hasWrittenFieldName && getFieldName() != null) {
                  generator.writeObjectFieldStart(getFieldName());
                 hasWrittenFieldName = true;
            }
            StructuredArgument structuredArgument = (StructuredArgument) argument;
            structuredArgument.writeTo(generator);
        } else if (includeNonStructuredArguments) {
              if (!hasWrittenFieldName && getFieldName() != null) {
                generator.writeObjectFieldStart(getFieldName());
                hasWrittenFieldName = true;
            }
          String fieldName = argument.getClass().getSimpleName();
            generator.writeObjectField(fieldName, argument);

My configuration of Decorator inside XML and the Decorator class is below

          public class CustomFactoryDecorator implements JsonFactoryDecorator{

    @Override
    public MappingJsonFactory decorate(MappingJsonFactory factory) {
        ObjectMapper codec = factory.getCodec();
        codec  = codec.copy()
                .setSerializationInclusion(JsonInclude.Include.ALWAYS)
                .setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY)
                .disable(SerializationFeature.FAIL_ON_EMPTY_BEANS)
                .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
                .configure(DeserializationFeature.FAIL_ON_IGNORED_PROPERTIES, false)
                .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);

        factory.setCodec(codec);
        return factory;
    }
}

Below is the XML config of encoder

           `<encoder class="net.logstash.logback.encoder.LogstashEncoder">
               <fieldNames><message>[ignore]</message></fieldNames>
              <jsonFactoryDecorator class="com.json.logging.CustomFactoryDecorator"/>
             <!-----CustomMessageProvider for logging raw messages-------> 
              <provider class="com.json.logging.CustomMessageJsonProvider"/>
               <provider class="com.json.logging.CustomArgumentsJsonProvider">
                    <includeNonStructuredArguments>true</includeNonStructuredArguments>
                </provider>
          </encoder>`

Logs are not formed inside the application after the event which throws this error. This error is due to the fieldName: value $Proxy244 . Basically the Proxy is set as We have an Annotation as below its an Annotation class @EmployeePersistProcessor which th aspectJ framework internally calls EmployeePersistProcessorImpl. Hence the name Proxy is returned. How to get the classnames if I get the class names in this scenarios

Also I get the same error if I apply

                String fieldName = nonStructuredArgumentsFieldPrefix + argIndex;
                generator.writeObjectField(fieldName, argument);

I get the error below

               `19:39:29,324 |-ERROR in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - IO failure in 
    appender com.fasterxml.jackson.databind.JsonMappingException: No serializer found for class 
   com.sun.proxy.$Proxy244 and no properties discovered to create BeanSerializer (to avoid 
    exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)
at com.fasterxml.jackson.databind.JsonMappingException: No serializer found for class 
    com.sun.proxy.$Proxy244 and no properties discovered to create BeanSerializer (to avoid 
  exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)
   at   at 
  com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:284)
at  at com.fasterxml.jackson.databind.SerializerProvider.mappingException(SerializerProvider.java:1110)
at  at com.fasterxml.jackson.databind.SerializerProvider.reportMappingProblem(SerializerProvider.java:1135)
at  at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:69)
at  at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:32)
at  at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:292)`

For this I have added below in the decorater but I get the below exception

              Objectmapper.disable(SerializationFeature.FAIL_ON_EMPTY_BEANS)

Again the below error is raised which stops the logging of the next events to be written inside the logfile.

error.com.fasterxml.jackson.databind.JsonMappingException: Infinite recursion (StackOverflowError) (through reference chain: java.util.concurrent.ConcurrentHashMap[" "]->com.java.sample.EmployeeDatabaseConfiguration$$EnhancerBySpringCGLIB$$6c9f7932["databaseProperties"]->com.java.sample.EmployeeDataConfiguration$$EnhancerBySpringCGLIB$$99c7d84b["$$beanFactory"]->org.springframework.beans.factory.support.DefaultListableBeanFactory["singletonObjects"]at at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:706) at at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:704) at at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:690) at at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:704) at at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:690) at at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at at com.fasterxml.jackson.databind.ser.std.MapSerializer.serializeFields(MapSerializer.java:633) at at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:536) at at com.fasterxml.jackson.databind.ser.std.MapSerializer.serialize(MapSerializer.java:30)

Please let me know what I am missing. I want everything in JSON format and I have tried arguments as above. Some of the messages do not get logged . How do I know that each and eve rything is logging properly with JSON format and what is the configuration required.**

    ` @Data      
     public class EmployeeDatabaseConfiguration{          
            private DatabaseProperties databaseProperties;       
      }`

What I have observed is that the classed which are not getting serialized have private fields, but other logs which are successfully written also contain private fields.

If ArgumentsJsonsJsonProvider.fieldName is not null, then all of the arguments will be serialized inside a sub-object of the main log event.

For example if fieldName is args, then the log event would be output as JSON like this:

{
  "@timestamp" : "2018-02-15T21:13:47.747-08:00",
  "@version" : 1,
  "message" : "3 Employee Object::::::",
  "logger_name" : "Main",
  "thread_name" : "main",
  "level" : "DEBUG",
  "level_value" : 10000,
  "args" : {
    "employeeString" : {
      "firstName" : "Sawyer",
      "lastName" : "Ford",
      "phoneNumber" : "12334433",
      "address" : "deddwde",
      "description" : "HireriepriwerierwerHellorwero"
    }
  }
}

Whereas, if the fieldName is null, then the log event would be output as JSON like this:

{
  "@timestamp" : "2018-02-15T21:13:47.747-08:00",
  "@version" : 1,
  "message" : "3 Employee Object::::::",
  "logger_name" : "Main",
  "thread_name" : "main",
  "level" : "DEBUG",
  "level_value" : 10000,
  "employeeString" : {
    "firstName" : "Sawyer",
    "lastName" : "Ford",
    "phoneNumber" : "12334433",
    "address" : "deddwde",
    "description" : "HireriepriwerierwerHellorwero"
  }
}

Please ensure you are using logstash-logback-encoder 4.11 to ensure you're not running into bugs that have already been addressed.

For example, in logback-logstash-encoder 4.8, two changes were made to addresses problems you have encountered.

First, SerializationFeature.FAIL_ON_EMPTY_BEANS is now disabled by default.
Second, if a different serialization error were to occur, the error will be logged to logback's status listener, but logging future events will continue to work. (i.e. an error during serialization will not cause logging to stop)

Other bugs have been fixed in later versions.

Regarding proxy objects... You can use AopUtils.isAopProxy(object) to determine if an object is a proxy, and AopUtils.getTargetClass(object) to get the class behind the proxy.

Regarding the Infinite recursion error.... It is important to realize that not every object can be serialized as JSON. Some objects have internal structures that prohibit it from being serialized properly. This is especially true for private member fields, which are sometimes never meant to be exposed outside of the object. For these objects/classes, you would need to configure Jackson with a custom serializer, or mix-in that ignores fields that can't be serialized, so that jackson can properly serialize the objects.

@philsttr : Now I am have up)graded to 4.11 ( my jackson version is 2.8.5 wud I have any issues, even tried with 4.9, 4.10). Now the exceptions are handled well but when the stackoverflowerror occurred the earlier logs before the stackoverflowerror are erased and vanished and the logs after the stackoverflow error are printed in the logs. Below is the error

08:04:50,948 |-WARN in net.logstash.logback.encoder.LogstashEncoder@350b47ad - Error encountered while encoding log event. OutputStream is now in an unknown state, but will continue to be used for future log events.Event: error.com.fasterxml.jackson.databind.JsonMappingException: Infinite recursion (StackOverflowError) (through reference chain: java.util.concurrent.ConcurrentHashMap[" "]->com.java.sample.EmployeeDatabaseConfiguration$$EnhancerBySpringCGLIB$$6c9f7932["databaseProperties"]->com.java.sample.EmployeeDataConfiguration$$EnhancerBySpringCGLIB$$99c7d84b["$$beanFactory"]->org.springframework.beans.factory.support.DefaultListableBeanFactory["singletonObjects"]at at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:706) at at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:704) com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:704) at at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:690) at at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155) at at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:704) at at

And moreover there will be thousands of classes/objects which might throw this error, How should I manage them with custom or mix-in serializers

If I have implemented custom serializes where should I add them is it to be inside the encoder ?

You are encountering the main problem with trying to serialize every argument as JSON. Jackson is not able to serialize every object due to things like infinite recursion in object graphs, like you are encountering.

My recommendation is to not try to serialize every argument as JSON. Again, because there is no guarantee that jackson will be able to serialize every type of object passed as an argument as JSON. You are extremely likely to encounter objects that are not JSON serializable by jackson, unless custom class-specific hints (such as JsonSerializers or mixin) are registered for the class.

Instead, I would recommend utilizing StructuredArguments. When StructuredArguments are used, it forces the developer to follow the contract of a StructuredArgument.... in that the argument must be JSON serializable. This forces developers to select which arguments it makes sense to try to JSON-serialize, and ensure those classes JSON-serializable.

However, if you would like to continue trying to serialize every argument, then you are going to have to come up with your own strategy. Most likely that strategy is not going to be able to magically make every object JSON-serializable. Therefore, you're going to have to make compromises.

For example, a simple strategy might be to whitelist classes that are known to be serializable, and not attempt to serialize classes that are not in the whitelist in your JsonProvider.

For classes that jackson cannot serialize by default, you can register mixins and custom serializers in a JsonFactoryDecorator. After including the appropriate mixin or custom serializer, you can add them to your whitelist.

You could also take the opposite approach, by having a blacklist... i.e. classes that are known to NOT be serializable. And only serialize classes that are not in that list. This would be more error prone than a whitelist approach, since new classes (that might not be serializable) are invented every day.

Another strategy would be to catch the exceptions from jackson in your JsonProvider, and dynamically add the classes that cannot be serialized by jackson to a list of blacklisted classes at runtime. (Therefore preventing them from being attempted to be serialized again).

There are probably other strategies that you can think of that will work for you.

The choice is yours to make. You will need to pick a compromise with which you are willing to live.

The logstash-logback-encoder library gives you the tools needed to implement your strategy. You can configure the JsonFactory, ObjectMapper, and JsonGenerator exactly how you want it with decorators. And you can create your own JsonProvider with full control over what to write to the JsonGenerator. You can experiment with Jackson, and find what options work best for you.

if i use log.error(Exception) , how can i further tokenize the message parameter to display a nested json for further individual fields?

logfellow / logstash-logback-encoder

The message attribute in the JSON log has backslash, new line and special charecters #264