vectordotdev / vector

A high-performance observability data pipeline.
https://vector.dev
Mozilla Public License 2.0
17k stars 1.47k forks source link

GELF decoder's default framer with TCP input does not match the encoder's default #20768

Open jorgehermo9 opened 6 days ago

jorgehermo9 commented 6 days ago

A note for the community

Problem

As discussed in #18008, the GELF encoder was using newline delimited but graylog expected it to be '\0' delimited (https://go2docs.graylog.org/5-0/getting_in_log_data/gelf.html)

Reviewing the code, it seems that the GELF decoder is still using the newline delimiter as default in TCP inputs https://github.com/vectordotdev/vector/blob/3de6f0b8175c8a08da1974d6b6e9634075244a06/src/sources/socket/mod.rs#L123 https://github.com/vectordotdev/vector/blob/3de6f0b8175c8a08da1974d6b6e9634075244a06/lib/codecs/src/decoding/mod.rs#L337

Should this be changed to match the encoder's default and read streaming gelf with the '\0' delimiter?

Configuration

No response

Version

0.40.0

Debug Output

No response

Example Data

No response

Additional Context

No response

References

18008

jszwedko commented 5 days ago

Thanks for filing this @jorgehermo9 . I tend to agree: the encoder and decoder should by symmetric. A concern I have is that GELF clients may be delimiting with newlines because that is what the spec calls for, but they can always override it. We'd welcome a PR fif you are up for it.

jszwedko commented 5 days ago

Actually, I stand corrected, the spec, as you linked to, does call for \0 delimited lines. Then yes, it seems like the decoder should definitely be using that by default.

jorgehermo9 commented 5 days ago

Thank you very much for your prompt response. I'm familiar with that code as I've been implementing #20769, and it is a very simple change. I'm pleased to submit a PR addressing this soon.