search

trentm / node-bunyan

a simple and fast JSON logging module for node.js services
Other
7.17k stars 515 forks source link

field "time" does not work with EFK or ELK stack #614

Open piggyman007 opened 5 years ago

piggyman007 commented 5 years ago

Hi all,

I am working on pipe the log to EFK stack using docker driver. I found that having the field "time", the log will disappear on kibana dashboard.

Can we allow 1 option to remove "time" field?

Or can we just change to another name like "timestamp"?

I don't have a chance to go through more detail and find the root cause yet.

piggyman007 commented 5 years ago

I create a PR https://github.com/trentm/node-bunyan/pull/615

Can someone please review it?

kerosan commented 5 years ago

same problem

trentm commented 4 years ago

(FYI on the delay in my responding: https://github.com/trentm/node-bunyan/issues/335#issuecomment-648583828)

This might have an answer as to the cause: https://stackoverflow.com/questions/59220299/fluentd-time-field-handling-in-json-log-records (basically that Kibana wants a particular format for "time" else it bumps the record time to 1970). I haven't used EFK or ELK myself.

For compat, I can't change the bunyan record format to avoid a "time" field, nor (at least for 1.x) change the time string format to match what Kibana wants to see. That could be considered for 2.x, but no promise there.

I think one might be able to use a "raw" Bunyan log stream to transform the "time" field on output, but I don't have the chance to check right now. E.g. see https://github.com/trentm/node-bunyan/blob/1.x/examples/raw-stream.js