This is a first pass at multiplexing data channels for source and sink operators.
I left a few TODOs to follow up on regarding:
Benchmarking DashMap against Moka or combining that with a simple hash map local to the multiplexer which retrieves from the concurrent map and stores the receiver based on (InstructionId, TransformId) pairs and evicting them when their last data/timers has been observed.
Using turmoil to simulate client and server environments in tests.
Additionally (not marked TODO), the mpsc channel is unbounded and that's fine for now since nothing's hooked up to it, so it's a little fuzzy to define a limit for a bounded channel.
laysakura
commented
1 year ago
This is a first pass at multiplexing data channels for source and sink operators. I left a few TODOs to follow up on regarding:
Additionally (not marked TODO), the mpsc channel is unbounded and that's fine for now since nothing's hooked up to it, so it's a little fuzzy to define a limit for a bounded channel.