Leverage Streams for Parallel Programming - E.g. Dataflow Programming / Functional Reactive Programming (FRP) / Communicating Sequential Processes (CSP) / Algorithmic skeleton (Parallelism Patterns)

[sirinath]

Libraries like http://www.paralleluniverse.co/quasar/, https://github.com/GPars/GPars, http://akka.io/ have this. It will be good to have similar features in this library also so you can have one cohesive library for similar needs.

[danieldietrich]

Thank you! Do you mean the feature of non-blocking streaming of data-chunks, which may be pulled from a source or may be also pushed from a source?

There has been some initial discussion in #79. At #186 I am tracking activities now.

[sirinath]

Something like this:

Val<Integer> a = new Val<>();
Var<Integer> x = new Var<>();
Var<Integer> y = new Var<>(() -> a.get() * x.get());
Var<Integer> z = new Var<>(() -> a.get() + x.get());
Var<Integer> r = new Var<>(() -> {
    int res = y.get() + z.get();
    System.out.println("res: " + res);
    return res;
});

Fiber<?> f = new Fiber<Void>(() -> {
    for (int i = 0; i < 200; i++) {
        x.set(i);
        Strand.sleep(100);
    }
}).start();

Strand.sleep(2000);
a.set(3); // this will trigger everything
f.join();

(Sourced from: http://docs.paralleluniverse.co/quasar/)

Also see documentation of: https://github.com/lihaoyi/scala.rx

[danieldietrich]

Without knowing how the classes are implemented this looks like some sophisticated implementation of the observer pattern with the addition of thread synchronization.

What I want is some few, strongly typed, composable building blocks. I haven't yet investigated what types of async abstractions are necessary to cover this whole topic with the minimum amount of types.

As said, without having deep-dived into that topic, the following dualities come to my mind:

• linear vs parallel (the latter splits into concurrent vs. distributed)
• push vs. pull
• blocking (sync) vs. non-blocking (async)
• ...

A good starting point is the reactive-streams project. Resources:

• Reactive Streams & Akka Http

Also non-functional aspects have to be covered, like

• automatic resource management (e.g. Autoclosables)
• automatic exception handling of composed things
• life-cycle management (when does information-flow end?)
• how to lift informations to processable entities (Chunks)
• ...

[sirinath]

I guess GPars is the most complete implementation. Perhaps you can look under the hood for what is happening.

[sirinath]

In the JS world this is a very interesting implementation: http://noflojs.org/

[danieldietrich]

out of scope for Javaslang