I need to read items from one place and write them to another. Both read and write operations will handle batches of items but the batch size will differ depending on the operation, e.g. I read 100 items at a time but can only write 25. I figured I could use streams in order to easily perform these operations in parallel but I can't find any operator that seems to fit my needs. All buffering operators seem to use a separator stream to control emits. I'm looking for something more like pairwise, only it should take an argument that sets the size of the buffer and also emits the remainder if the source completes. There's probably a way to accomplish this by composing operators but all examples I've seen use interval based streams to control the buffer.
I need to read items from one place and write them to another. Both read and write operations will handle batches of items but the batch size will differ depending on the operation, e.g. I read 100 items at a time but can only write 25. I figured I could use streams in order to easily perform these operations in parallel but I can't find any operator that seems to fit my needs. All buffering operators seem to use a separator stream to control emits. I'm looking for something more like
pairwise, only it should take an argument that sets the size of the buffer and also emits the remainder if the source completes. There's probably a way to accomplish this by composing operators but all examples I've seen use interval based streams to control the buffer.