new $TRANSPORT macro in 4.5.0 gives unexpected results

[czanik]

I wanted to give $TRANSPORT in 4.5.0 draft release a try. I set up various sources and checked the value of $TANSPORT:

• legacy syslog over UDP: "TRANSPORT":"rfc3164+udp"
• IETF syslog over TCP: "TRANSPORT":"rfc6587"
• legacy syslog over TCP: "TRANSPORT":"rfc3164+tcp"

However, if I use the following source:

source net {
  network(transport(tcp) port(6666) flags(syslog-protocol));
};

It sitll says "TRANSPORT":"rfc3164+tcp", even if it is rfc5424 syslog, just without octent counting. This way it cannot be distinguished from RFC3164 TCP connections.

[HofiOne]

This also relates to the ongoing https://github.com/syslog-ng/syslog-ng/pull/4544 refactor

Can someone please give me a proper cross-reference table of all the possible combinations we can count on here? I'm interested mainly in the RFC names, it would be a great help both for the testing and for #4544 TIA!

[bazsi]

The content counting IS the transport, not the message format. We accept many formats both via the network() and the syslog() sources.

syslog() accepts both RFC3164 and RFC5424 by default. network() accepts RFC3164 but you can enable RFC5424 too via the flag as you instructed it to do.

Transport is about well the transport protocol and not the message format.

The reason we do "rfc3164+tcp" in the case of legacy log is that in the case of RFC3164 the transport and message format were not specified separately, like in the case of the rfc5424 family, in which case you have:

rfc5424 is the message format, independently from transport

• rrc5425 is the TLS transport
• rfc5426 is the UDP transport
• rfc6587 is the octet-counted TCP transport.
• we have one transport value for custom transports, rfc5424+ for something like altp

In case of rfc3164, we have the following values for $TRANSPORT:

• rfc3164+udp
• rfc3164+tcp
• rfc3164+tls
• rfc3164+proxied-tls
• and we have rfc3164+, this would apply to something like altp

[bazsi]

I would close this as "works as intended", we were discussing another macro that could potentially describe the original format that was coming on top of $TRANSPORT, which could be $MSGFORMAT for example, but that's outside of the scope of the original PR.

And with all the heuristics around recognizing various logging formats, $MSGFORMAT would just be a guess anyway.

[czanik]

Well, I find it a bit confusing, but can live with it...

[bazsi]

How would you set $TRANSPORT if we were reading the rfc5424 messages from a file, a unix domain socket or through OTLP.

What it you accept rfc3164 on the same transport?

Like I said transport is about the application protocol we use to transmit messages in whatever format. There are multiple potential formats that the same protocol can host.

On Sat, Dec 2, 2023, 17:07 Peter Czanik @.***> wrote:

Well, I find it a bit confusing, but can live with it...

— Reply to this email directly, view it on GitHub https://github.com/syslog-ng/syslog-ng/issues/4731#issuecomment-1837191260, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFOK5S27W33ASY5WJ3MD5DYHNG25AVCNFSM6AAAAAA74FVWGGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZXGE4TCMRWGA . You are receiving this because you commented.Message ID: @.***>

[czanik]

OK. Just did a quick test:

source net {
  syslog(transport(tcp) port(6666));
};
destination fromnet {
  file("/var/log/fromnet" template("$(format-json --scope everything)\n\n"));
  file("/var/log/fromnet.txt");
};
log {source(net); destination(fromnet);};

And on the sender side I used:

logger --rfc3164 -T --octet-count -n 172.16.167.174 -P 6666 this is a test

And the result:

"TRANSPORT":"rfc6587"

So yes, it also works the reverse way.

However, it also means that $MSGFORMAT could resolve lots of confusion. Closing this for now.