Abstractions for sharing messages across channels

shsms commented 3 months ago

What's needed?

There seem to be cases where it would be useful to link multiple channels - either sending the same message into multiple channels, or sending all messages received by a channel into another channel.

Proposed solution

Implement:

TeeSender, like the tee command. Implements Sender, accepts multiple senders as constructor parameters, and sends messages sent to it to each of the destination senders.
Bridge that takes a sender and a receiver, and send messages to the sender that it received from the receiver.

Use cases

No response

Alternatives and workarounds

No response

Additional context

No response

llucax commented 3 months ago

If we have a merge function, maybe we should name the TeeSender a Splitter? Following this analogy maybe Connector could be more intuitive than Bridge too.

But besides the bike shedding, I wonder about what use cases we have though, as we've been removing some code without use cases before the 1.0.0 release, I think it would be good to make sure we have valid use cases for this.

Why would you need to do this though, instead of just using a Broadcast channel and have multiple receivers instead, or just sending directly to the end sender instead of having an extra channel in the middle? I guess it makes sense only when you don't have control over the channels (you need to send to channels created by someone else). If we have real uses cases for this it would be good to put them here.

shsms commented 3 months ago

maybe we should name the TeeSender a Splitter

Splitter doesn't work, because there's no splitting happening, it is sending each message to all senders. Maybe there are other options.

Connector could be more intuitive than Bridge too.

Maybe. or Relay.

I guess it makes sense only when you don't have control over the channels (you need to send to channels created by someone else)

This is a major part of it, because according to our idioms, channels used to communicate with actors are never owned by the actors. Combining that with nested actors, there are often cases where we want send some data both upstream, outside an actor, and in the same level, to a different sub-actor.

Other than that, because these are only operating on senders and receivers, it can be used for things like sending a FileWatcher event or Timer event to multiple receivers, but bridging them to a channel.

I wonder about what use cases we have though, as we've been removing some code without use cases before the 1.0.0 release, I think it would be good to make sure we have valid use cases for this.

We removed MergeNamed because it was duplicate, and we removed Peekable, but replaced it with LatestValueCache, because it was less intrusive.

I think both these new proposed features enable new ways for composing channels, when you have just a Sender or Receiver to begin with, and don't get in the way of other features.

We can take longer to decide of course. Until then, I can implement custom solutions in places where this would be useful.

cwasicki commented 3 months ago

Other options discussed:

Connector Relay.

Append, Joint, Extend.

Splitter

Combined, MultiSender, TeeJoint

llucax commented 3 months ago

We can take longer to decide of course. Until then, I can implement custom solutions in places where this would be useful.

I'm playing devil's advocate here mainly :upside_down_face:, I have no major issues with it. I see use cases for it, I'm just concerned that we could end up having people starting to patch senders and receivers together without thinking much about it and adding unnecessary complexity, but any tool can be abused, I guess we just have to be careful and try to detect (and avoid) these cases as soon as possible.

Splitter doesn't work, because there's no splitting happening, it is sending each message to all senders. Maybe there are other options.

That's what a splitter does (I mean in the sense of a telephony line splitter, video signal splitter, audio splitter, etc.).

About the names, when asking to ChatGPT it also suggested MultiSender, for me Splitter is much more obvious in the context, but ChatGPT being the average of what people in the Internet think, maybe it is a good choice even if I don't like it.

Maybe. or Relay.

Realy seems good but I think it might be confusing given that unfortunately is a term that is it widely used in the electricity world with another meaning. Another suggestion by ChatGPT is Forwarder.

About Append and Extend I'd rather not use verbs as class names. Joint could also be an option, but for me it lacks a bit the feeling of information being passed in one direction, which I think is more clear with other alternatives.

I find Combined backwards, for me it sounds more like Merger and for TeeJoint I have the same complaints as with TeeSender.

I would really prefer not to use tee in the name as it will be completely unintuitive to someone not coming from the unix world and Relay because it could be very confusing when using in the microgrid context, and similar with Combined, as think it might be confusing when we already have a Merger.

I think my vote goes to Splitter and Forwarder, but I can live with MultiSender and Bridge/Connector too. Less happy about Joint, but I could live with it too.

shsms commented 3 months ago

That's what a splitter does (I mean in the sense of a telephony line splitter, video signal splitter, audio splitter, etc.).

I think a splitter means something completely different in a context with discrete messages, than what it means with analogue signals.

Realy seems good but I think it might be confusing given that unfortunately is a term that is it widely used in the electricity world with another meaning

I think that's not an issue, the meanings are similar, even in the electricity context. Just like we chose "Merger" even if the only definition you can find on the internet is related to mergers and acquisitions. But we can also call it "Relayer" or "RelaySender".

and Bridge/Connector too

I think Bridge is a Connector too. It is a connector specifically made to facilitate the movement of traffic. I think any of "Bridge/Tunnel/Pipe" is better than just Connector.

cwasicki commented 3 months ago

To me these don't sound like very unusual problems, so I wonder how this is solved with channel implementations in other languages so we could follow the terminology there. Couldn't find something about this easily though.

Not sure whether this makes sense, but another option could be to create a detached receiver which can be passed around and then be attached to an existing channel?

About Append and Extend I'd rather not use verbs as class names.

This could be solved, e.g. Appender

I would really prefer not to use tee in the name as it will be completely unintuitive to someone not coming from the unix world

Actually also for people somewhat familiar with unix I have to say.

Combined, as think it might be confusing when we already have a Merger.

Combined -> CombinedSender.

I think my vote goes to Splitter and Forwarder, but I can live with MultiSender and Bridge/Connector too.

Same. And Pipe :+1:

I think a splitter means something completely different in a context with discrete messages, than what it means with analogue signals.

I think we need a new theory.

So current candidates are the following?

1) Bridge, Forwarder, Tunnel 2) TeeSender, MultiSender, Splitter

llucax commented 3 months ago

Sorry, this got a bit buried and I just found it again.

Relayer sounds good to me too. I'm starting to like more agent nouns than plain nouns for this.

To make it short, my most viable candidates are:

Relayer, Forwarder. I could live with Bridge, Pipe, Tunnel or Joint but I think those sound more bidirectional, I like from the former that give a better sense of one-directionality.
MultiSender seems to be the only option we all agree about. ChatGPT insists the action of sending the same message to multiple streams is broadcasting, but I think it would be too confusing to have a Broadcaster and a Broadcast channel that are not related. I also offers muticast as an option (Multicaster? MulticastSenser?) which could be an option too I guess.

From my side, I think it will be hard to find better names than these as we've been discussing for a while, so I'm OK if Sahas (as the PR author) decides. I'm OK with any of the options above, and I would strongly prefer not to use Tee*.

cwasicki commented 3 months ago

Pipe, ... but I think those sound more bidirectional

I don't think pipe would be considered bidirectional but is typically used in unidirectional contexts, e.g. unix pipes, ETL pipeline. In fact this would be my favorite to describe this concept.

frequenz-floss / frequenz-channels-python