At the moment, any data sent through a Timely stream needs to be Clone because Tee needs to be able to clone data to send it to multiple recipients. This change tries to change this, although it comes with far-reaching and breaking changes.
I don't intend to merge this yet, but it seems like a solid basis for further experiments.
Specifically, it introduces new types and changes behavior:
Operators by default produce an OwnedStream that can only be connected to one downstream operator. Because of this, there's no Tee involved that would require data to be Clone.
To force a Tee, OwnedStream has a tee function that returns a stream that can be shared between downstream operators. This function requires the data to be Clone and just like all other operators consume the stream.
Operators are implemented for a StreamLike type to be generic over OwnedStream and StreamCore. This requires a few more type parameters here and there.
It removes the Clone requirement from Container. This adds a bound +Data requirement to all operators that want to take ownership of their container input (which is all operators.)
The names are .. not great. Open to suggestions.
The effect of this change is that for op --> op structures, we only have a vcall where we currently have a vector dereference plus vcall. This can be better in some situations, but I didn't measure it. The downside is that requesting a tee adds the vector dereference plus vcall after the first vcall, so it's strictly worse. This should be amortized by how infrequently it's used, but who knows.
In theory, we don't need the vcall at all because the type of the downstream operator is known to Rust and hence its pusher. However, I currently can't see how to wire it up, and I believe it's ugly because the information needs to flow backwards from the receiving operator to the source.
At the moment, any data sent through a Timely stream needs to be
ClonebecauseTeeneeds to be able to clone data to send it to multiple recipients. This change tries to change this, although it comes with far-reaching and breaking changes.I don't intend to merge this yet, but it seems like a solid basis for further experiments.
Specifically, it introduces new types and changes behavior:
OwnedStreamthat can only be connected to one downstream operator. Because of this, there's noTeeinvolved that would require data to beClone.Tee,OwnedStreamhas ateefunction that returns a stream that can be shared between downstream operators. This function requires the data to beCloneand just like all other operators consume the stream.StreamLiketype to be generic overOwnedStreamandStreamCore. This requires a few more type parameters here and there.Clonerequirement fromContainer. This adds a bound+Datarequirement to all operators that want to take ownership of their container input (which is all operators.)The effect of this change is that for
op --> opstructures, we only have a vcall where we currently have a vector dereference plus vcall. This can be better in some situations, but I didn't measure it. The downside is that requesting ateeadds the vector dereference plus vcall after the first vcall, so it's strictly worse. This should be amortized by how infrequently it's used, but who knows.In theory, we don't need the vcall at all because the type of the downstream operator is known to Rust and hence its pusher. However, I currently can't see how to wire it up, and I believe it's ugly because the information needs to flow backwards from the receiving operator to the source.