Transformer.orderedMergeWith: Allow configuring Source Observable Backpressure BufferSize

[moderakh]

Hi David,

We are using rxjava-extras as a dependency: https://mvnrepository.com/artifact/com.microsoft.azure/azure-cosmosdb/1.0.0 https://github.com/Azure/azure-cosmosdb-java

We are using Transformer.orderedMergeWith. It appears to me that in the implementation of BackPressure for the above operator for each of the source observables, RxRingBuffer.SIZE items will be buffered. Am I right? https://github.com/davidmoten/rxjava-extras/blob/08e66ff7ab2ef530292ff1ee1e1f4827aafee08e/src/main/java/com/github/davidmoten/rx/internal/operators/OrderedMerge.java#L307

If not already supported, is that possible to provide a overload for Transformer.orderedMergeWith which allows configuring the buffered size for each source observable? It is critical for our use case.

Thank you @davidmoten

[davidmoten]

@moderakh That won't be a problem. I'll knock that up.

[davidmoten]

@moderakh when you say it's critical for your use case, can you be more specific?

[moderakh]

@davidmoten Azure Cosmos db Orderby is computationally expensive and also each user is assigned a fixed budget for query in every second.

When a query in user's scope is executed we deduct user's budget. Right now for our cross partition orderby query in the SDK side we are relying on Transformer.orderedMergeWith.

The fact that Transformer.orderedMergeWith fetches a lot of data upfront means the computational expense for running orderby query and fetching a lot of data will be paid upfront and even if the user is only interested in the first few pages of results still not having a configurable buffer size means the user will have to pay some additional cost upfront.

[davidmoten]

Thanks for the explanation, makes sense. 0.8.0.15 has the new overload, should be available for download from Maven Central in about ten minutes.

[moderakh]

wow, that's amazing. Thank you for such quick feature implementation and release. :-)