vectordotdev / vector

A high-performance observability data pipeline.
https://vector.dev
Mozilla Public License 2.0
17.68k stars 1.56k forks source link

Add `connect` mode to the `socket` source #2042

Open ghost opened 4 years ago

ghost commented 4 years ago

Currently the socket source uses the specified socket address (for TCP/UDP) or path (for UDS) to create a socket with such an address and listen for connections on it.

However, in some circumstances it might be useful to not listen, but connect to sockets and read data from them.

I propose to add new configuration option, which would take two possible values, listen (the current behavior) and connect (the new mode). I'm not sure what would be the best name for the new configuration option. It could have been mode, but mode is already used to specify whether the socket is TCP/UDP/UDS.

binarylogic commented 4 years ago

@bruceg I'm curious what you think about this, and how much work it would be. I'm wondering if this is better solved with an entirely different source?

bruceg commented 4 years ago

From a user point of view, I think it makes sense to put it into the same module. From a code point of view, it will require some significant reorganization. However, since the socket source is already a module, it is partially set up for this already.

I'd like to say it should be a quick conversion, since the core protocol is trivial and unchanged, but the connection-based wrapping would be completely different. Should definitely be doable within a sprint though.

FTR, this mode only makes sense for TCP and UDS modes, since UDP doesn't have connections as such.

binarylogic commented 4 years ago

Thanks. Let's hold off for the time being. I'm curious how much demand we have for this.

jszwedko commented 4 years ago

Noting this was requested in Discord: https://discordapp.com/channels/742820443487993987/746070591097798688/755065974360899604

jszwedko commented 3 years ago

Relevant comment on another issue: https://github.com/timberio/vector/issues/2041#issuecomment-734364233

fatihusta commented 5 months ago

Is there any update for this issue?