JSON Extractor doesn't work with ESET Syslog

[qarmin]

Hi, I'm tried to extract JSON message sended by ESET with builtin JSON Extractor, but I'm unable to do this.

I'm getting from Eset ESMC syslogs log which have in message field this {"event_type":"Audit_Event","ipv4":"www","hostname":"asfasfaas","source_uuid":"fase4b96","occured":"29-Jan-2020 09:50:07","severity":"Information","domain":"Domain group","action":"Logout","target":"wfe56615b","detail":"Logging out domain user 'asf.asf'.","user":"asf.asf","result":"Success"}

This message should be properly extracted to multiple fields, but I have each time only Nothing will be extracted message.

Steps to Reproduce (for bugs)

1. Configure Eset 7 ESMC to send JSON Syslogs
2. Create input in Graylog
3. Wait to first log message
4. Try to create JSON extractor(Just try to get properly text when clicking at "Try" button)

Your Environment

• Graylog Version: 3.2.0-3.beta.3
• Elasticsearch Version: 6.8.6-1
• MongoDB Version: 4.0.14-1.el7
• Operating System: CentOS 7
• Browser version: Firefox 72.0.2

[kmerz]

I tried to reproduce your problem with the sample data you provided and it worked:

Could you please try it with the sample data (with a simple GELF or TCP input). If that works, can you please sample data more closely to your acutal data? Perhaps the problem is somewhere in the blacked out strings?

[qarmin]

How can I send manually messages to my server to test different messages and configuration? I read manual, but I'm only found example with CURL and GELF which doesn't work for me

curl -XPOST http://graylog.example.org:12202/gelf -p0 -d '{"short_message":"Hello there", "host":"example.org", "facility":"test", "_foo":"bar"}'

[qarmin]

Can you try to check this JSON? It is almost identically as original

{"event_type":"Audit_Event","ipv4":"182.12.1.242","hostname":"romna.sqe.feg","source_uuid":"5ab3459d-d6e5-480b-8a1e-8811d62e4b96","occured":"30-Jan-2020 12:42:53","severity":"Information","domain":"Domain group","action":"Login attempt","target":"763d4a3f-98fe-4697-93a4-5fd36e73615b","detail":"Authenticating domain user 'roman.koudelka'.","user":"","result":"Success"}

[kmerz]

This is how I send the message to my gelf http input listening on port 12201:

curl -XPOST http://localhost:12201/gelf -p0 -d '{"short_message":"{\"event_type\":\"Audit_Event\",\"ipv4\":\"182.12.1.242\",\"hostname\":\"romna.sqe.feg\",\"source_uuid\":\"5ab3459d-d6e5-480b-8a1e-8811d62e4b96\",\"occured\":\"30-Jan-2020 12:42:53\",\"severity\":\"Information\",\"domain\":\"Domain group\",\"action\":\"Login attempt\",\"target\":\"763d4a3f-98fe-4697-93a4-5fd36e73615b\",\"detail\":\"Authenticating domain user 'roman.koudelka'.\",\"user\":\"\",\"result\":\"Success\"}", "host":"hulud.net", "facility":"test", "_foo":"bar"}'

And here the JSON Extractor result:

Do you see anything in your graylog server logs?

[qarmin]

gelf http input

I was using gelf http tcp, and that is why test messages not working.

So, Gelf HTTP works perfectly, I don't need even enable Flatten Structures, because everything extract correctly. But with Syslog UDP, I still can't use JSON extractor.

I tested it on two exactly same messages, so seems that something is wrong with Syslog UDP

[qarmin]

This also doesn't work with Syslog TCP

[kmerz]

Hello @qarmin,

the input does not matter. The content of the field is only important:

This is the result from a Syslog UDP Input.

I had problems in the beginning since the json was not escapted correctly. You can try it yourself with this:

logger '{"event_type":"Audit_Event","ipv4":"182.12.1.242","hostname":"romna.sqe.feg","source_uuid":"5ab3459d-d6e5-480b-8a1e-8811d62e4b96","occured":"30-Jan-2020 12:42:53","severity":"Information","domain":"Domain group","action":"Login attempt","target":"763d4a3f-98fe-4697-93a4-5fd36e73615b","detail":"Authenticating domain user 'roman.koudelka'.","user":"","result":"Success"}'

[qarmin]

I checked and logs from logger(Syslog UDP and TCP) can be easily parsed by JSON Now I don't know what can be wrong.

Maybe full messages can be helpful?

Full message from Eset

<14>1 2020-01-31T11:45:20.465Z server ERAServer 1142 - - {"event_type":"Audit_Event","ipv4":"11.11.11.11","hostname":"server","source_uuid":"12521ee-d6e5-480b-8a1e-8811d31e4b96","occured":"31-Jan-2020 11:45:20","severity":"Information","domain":"Domain group","action":"Login attempt","target":"22425124-98fe-4097-93a4-5fd36e56615b","detail":"Authenticating domain user 'user.user'.","user":"","result":"Success"}

Full Message from Logger

<13>1 2020-01-31T12:51:13.824388+01:00 graylog user - - - {"event_type":"Audit_Event","ipv4":"11.11.11.11","hostname":"server","source_uuid":"12521ee-d6e5-480b-8a1e-8811d31e4b96","occured":"30-Jan-2020 13:13:30","severity":"Information","domain":"Domain group","action":"Logout","target":"22425124-98fe-4097-93a4-5fd36e56615b","detail":"Logging out domain user user.user.","user":"user.user","result":"Success"}

[kmerz]

@qarmin how do you extract the json part from the message?

[qarmin]

I already have message showed as JSON by default(I don't change anything)

Message from top is from ESET and from below it comes from Logger centos command

[kmerz]

Can you take a look in your graylog server.log file and look for extractor errors when click on Try? Also can you check if any other extractor is configured on that input? Sometimes the output of one extractor can make it impossible for the JSON parser to read the string.

[qarmin]

I created new a new input only for Eset and restated server but still JSON extractor not working and nothing is printed to /var/log/graylog-server/server.log when I click at Try button

[qarmin]

I found a workaround, and probably cause of this problem As workaround I enabled Full Messages and created Regex extractor to extract all JSON in format {json:json} - I used this regex for search [^\{]+\{ and this for replace \{ And JSON extractor for this works perfectly

I copied to WineMerge text from 'message' and 'jsonFix'(created by me in previous step) an I found that this messages are different

I copied text {"event_type" to site https://www.browserling.com/tools/text-to-ascii and I got different results for each message

• Default message - 239 187 191 123 34 101 118 101 110 116 95 116 121 112 101 34
• JsonFix - 123 34 101 118 101 110 116 95 116 121 112 101 34

So at the begging there are 239 187 191 which seems to broke JSON Extractor.

[kmerz]

@qarmin that is a really helpful information! It makes sense that JSON can't parse this strange character. I had a glimpse on ESET and saw that some of their applications support different SYSLOG fromats. Could you try another one? And to document the behavior here, what is currently selected?

[qarmin]

There are several options in Eset, but seems that nothing works

• [x] - means that I tested it

Format

• [x] Syslog
• [ ] BSD - Graylog doesn't seems to support this

Transport

• [x] TCP
• [x] UDP
• [ ] TLS

Octet-counted framing

• [x] ON
• [x] OFF

Exported logs format:

• [x] JSON
• [x] LEEF - Logs are very strange and hard to parse, and still contains this strange numbers

Eset settings page

[kmerz]

As far as I know we so support BSD formatted syslog messages. Just for the sake of completion of this Issue could you try BSD as well?

Otherwise you are right with everything you wrote graylog has problems converting json if any content precedes the first { character.

Another workaround would be to create your extractors as pipeline rules.

But the actual bug is hard to fix without a way to have the ESET server running for our self. So I will leave this bug open as a reminder.

[qarmin]

Eset doesn't add this strange characters to BSD logs, but still I can't use JSON Extractor default message, because I have before first { - "eset..... ERAServer[1144]: "

[qarmin]

This numbers seems to be BOM of utf-8 - https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8 I fill bug report to Eset, but I get info that formatting of syslog depends on the Syslog server.

[maniel]

i noticed it sometime ago, i asked on Graylog Comunity about it here, it seems like ESET sends BOM at the beginning of the json message and it breaks Graylog extractor, the workaround i found out is included in the linked thread