ReactiveX / RxJava

RxJava – Reactive Extensions for the JVM – a library for composing asynchronous and event-based programs using observable sequences for the Java VM.
Apache License 2.0
47.87k stars 7.6k forks source link

Making Joins Dynamic #1329

Closed sirinath closed 10 years ago

sirinath commented 10 years ago

Current Joins implementation is very much static. This might need re thinking to make this more dynamic as new stream connections open and close. May be this can be done in multiple stages 0) Bring Rx Java in parity with Rx 1) Improve Current Static version 2) Build basic optimiser knowing the streams will not change 3) Add dynamic framework. By now anything done with the rest of Rx should be doable within the concurrency "Joins" subset 4) Build a optimiser for the dynamic case 5) Add coordination capabilities between optimisers 6) Build a distributed optimiser perhaps bringing Joins to RxNetty 7) Implement the old Rx contracts on top of this to take advantage of the optimiser and to port old code to the new framework. (Need not be seamless but less code changes.)

In Netflix context if the dynamic version is build this can be used for 1) Can be the basis of CEP engine 2) Build and dynamically optimise the compositions, transformation, dataflow graph for the full reactive infrastructure (fingers crossed might be orders magnitude faster than hand coded combinations and compositions. Some what relate (impliemtns Joins) project has achived impressive performance boost https://github.com/VesaKarvonen/Hopac. Also see http://stratosphere.eu/, Dryad, others in the family: http://en.wikipedia.org/wiki/Join-calculus, Cω by headinthebox, et al)

Also borrow heavily from other Process Algebra / Calculi and Trace Theory related research if Joins are for some reason not enough.

benjchristensen commented 10 years ago

@sirinath What is your desired outcome of this issue? /cc @headinthebox

Based on the variety of issues you have filed about joins, are you interested in contributing to the rxjava-joins (https://github.com/Netflix/RxJava/tree/master/rxjava-contrib/rxjava-joins) module?