graylog-labs / graylog2-web-interface

[DEPRECATED]
https://www.graylog.org/
611 stars 174 forks source link

Indexing issues with ElasticSearch while using graylog #1721

Closed sandeepky closed 8 years ago

sandeepky commented 8 years ago

Hi, I am using graylog to receive logs from my nodejs application- Node App -> Winston logger -> File on Disk -> NXLog -> Graylog Server

This is working perfectly. The messages NXLog is sending have the following format: {"app":"OpsPanel","func":"update","success":"true","level":"info","message":"logging service","timestamp":"2016-01-09T08:05:10.576Z"}

Graylog parses it and elasticsearch indexes the level as an integer, 6 in this case. The rest of the logged message remains as a json string.

I want to extract all the fields from the json message and index them. So, I use an extractor. The extractor, on TRY, shows the result exactly the way I want it to. However, it parses level and there is a string value, "info" in this case. However, ES expects an integer and this results in Indexer failure.

My question is- when the system is able to gauge the level as an integer initially, what stops it from doing that after the extraction?

joschi commented 8 years ago

Graylog parses it and elasticsearch indexes the level as an integer, 6 in this case.

I suspect this is simply the default level chosen by nxlog. If the message was logged on another log level (e. g. "error"), the level in the GELF message would still be '6'.

FWIW, there's a GELF appender for Winston which you could use to send those messages directly into Graylog: https://www.npmjs.com/package/winston-graylog2

In the future, please direct support questions (i. e. no bug reports or feature requests) to the mailing list or join #graylog on Freenode.