Gelf chunk message ID is not unique

[VictorSlavov]

I'm testing the library in a busy environment where we have around 10 JVMs on a single vmware VM and around 30 VMs total. This generates about 8k messages per second. The last 4 bytes of the message ID of a gelf chunk is formed by last 4 bytes of the host name, but in a standard environment many hosts would be in the same domain, hence for all hosts the last 4 bytes would be the same for example ".com", ".foo", etc. In my environment these are 300 JVMs setting the same last four bytes the same for their chunks. In that regard I would prefer to avoid using "originHost" parameter to try to workaround the issue. About the first four bytes, these are formed by the currentTimeMillis and there are many cases where two different threads would get the same time to generate the message ID. Bottom line is that there is quite a big chance to generate the same message ID for two different messages which will make graylog server to drop the messge(s). The duplicate chunk metric for last 12 hours on my graylog server is 2141. I was thinking of introducing some more random way of generating those message IDs. Do anyone have any ideas of what would be a better way (than the current) to do it?

[VictorSlavov]

I've changed message id generation like this and will test with it. in class GelfMessage:

    byte[] messageId = new byte[8];
    new Random().nextBytes(messageId);

[t0xa]

@VictorSlavov looking at the GELF spec this is valid messageId. How is your testing going?

[VictorSlavov]

Hey t0xa, Yes it's valid, yet not useful with many VMs sending large messages split in chunks through UDP. The test was successful. With randomly generating message ids for chunked messages I have no duplicates any more.

[t0xa]

Version 1.1.14 with fixed released.