OpenLiberty / open-liberty

Open Liberty is a highly composable, fast to start, dynamic application server runtime environment
https://openliberty.io
Eclipse Public License 2.0
1.16k stars 592 forks source link

missing fields from JSON logs #12391

Closed donbourne closed 4 years ago

donbourne commented 4 years ago

Describe the bug intermittently we see some records are missing fields from the JSON log. In the following block I have added spaces to emphasize the missing fields. In the 4th row of output the host, ibm_userDir and ibm_serverName fields are missing.

{"type":"liberty_trace","host":"acmeair-flightservice-6bb498d95c-czt7w","ibm_userDir":"\/opt\/ibm\/wlp\/usr\/","ibm_serverName":"defaultServer","message":"Search for flights from CDG to YUL on Mon May 25 00:00:00 GMT 2020",                            "ibm_threadId":"00000b2d","ibm_datetime":"2020-05-13T18:08:19.781+0000","module":"com.acmeair.service.FlightService","loglevel":"FINE",  "ibm_sequence":"1589393299781_0000000044510","ext_thread":"Default Executor-thread-2788"}
{"type":"liberty_trace","host":"acmeair-flightservice-6bb498d95c-czt7w","ibm_userDir":"\/opt\/ibm\/wlp\/usr\/","ibm_serverName":"defaultServer","message":"flightSegmentIdAndScheduledDepartureTimeQueryString AA379Tue May 19 00:00:00 GMT 2020",         "ibm_threadId":"00000b4b","ibm_datetime":"2020-05-13T18:08:19.781+0000","module":"com.acmeair.service.FlightService","loglevel":"FINE",  "ibm_sequence":"1589393299781_0000000044511","ext_thread":"Default Executor-thread-2818"}    
{"type":"liberty_trace","host":"acmeair-flightservice-6bb498d95c-czt7w","ibm_userDir":"\/opt\/ibm\/wlp\/usr\/","ibm_serverName":"defaultServer","message":"Returning []",                                                                                  "ibm_threadId":"00000b4b","ibm_datetime":"2020-05-13T18:08:19.781+0000","module":"com.acmeair.service.FlightService","loglevel":"FINEST","ibm_sequence":"1589393299781_0000000044513","ext_thread":"Default Executor-thread-2818"}
{"type":"liberty_trace",                                                                                                                        "message":"Segment {\"_id\": \"AA279\", \"originPort\": \"CDG\", \"destPort\": \"YUL\", \"miles\": 3433}", "ibm_threadId":"00000b2d","ibm_datetime":"2020-05-13T18:08:19.781+0000","module":"com.acmeair.service.FlightService","loglevel":"FINE",  "ibm_sequence":"1589393299781_0000000044512","ext_thread":"Default Executor-thread-2788"}
{"type":"liberty_trace","host":"acmeair-flightservice-6bb498d95c-czt7w","ibm_userDir":"\/opt\/ibm\/wlp\/usr\/","ibm_serverName":"defaultServer","message":"Segment in JSON {\"_id\":\"AA279\",\"originPort\":\"CDG\",\"destPort\":\"YUL\",\"miles\":3433}","ibm_threadId":"00000b2d","ibm_datetime":"2020-05-13T18:08:19.781+0000","module":"com.acmeair.service.FlightService","loglevel":"FINE",  "ibm_sequence":"1589393299781_0000000044514","ext_thread":"Default Executor-thread-2788"}

searching for this problem in logs can be done as follows:

oc logs <pod> | grep -v "ibm_serverName"

Expected behavior expect the host, ibm_userDir and ibm_serverName fields to be present on all liberty_trace events

Diagnostic information:

Additional context Add any other context about the problem here.

ellen-lau commented 4 years ago

Hasn't been reproducible in tests with later fixpack versions after the introduction of the new JSONBuilder class, so closing for now.