joschi
commented
6 years ago @xose Thanks for opening this issue!
Similar to #21, the GELF reader classes are not meant for public consumption. They are only used in the test cases for the GELF client implementations.
The focus of go-gelf is providing a GELF client, not a server.
/cc @mariussturm
Under high load, the UDP Reader will drop messages, corrupt a sizeable part of those that arrive, and even panic.
The issues are caused by GELF chunks arriving out of order or mixing different messages. The Reader drops some messages that arrive out of order. Others will be merged with chunks from different sources, causing corruption that can either crash the JSON parser (due to malformed JSON) and thus a dropped message, or return valid JSON but with corrupt contents to the user. Sometimes it will even result in a panic, crashing the application.
I wrote a test case to illustrate the issues here: https://github.com/xose/go-gelf/blob/stress/gelf/reader_test.go
The test case starts an UDP Reader and creates several goroutines to simulate multiple sources sending a large amount of GELF messages to the server. It then reads those messages and verifies their integrity.
Each "log" contains a long string (of random length) of the same character, different for each source, and extra fields with the character and expected message length. These values are used to check GELF message integrity.
The GELF messages are constructed by the test case instead of using the library itself to rule out an issue on other parts of the library. For GELF messages longer than a threshold the message is chunked. No compression is used. Test values can be configured changing the constants at the top of the file.
All errors will be logged on the console, and a summary will be printed at the end, like this: