CSP channels

[getify]

I would be very interested to see where CSP channels (ala go, clojure's core.async), which have been ported to JS (like here, here, and even here), fit into this unified theory.

At first brush, it would seem they are a variation on pressured streams. But I'm not convinced that's all they are, because there are so many variations on how the channels can be consumed (take, takem, alts, etc), as well as the composability of them implied by transducers.

[kriskowal]

This requires further study. Thanks for the links. The appeal of transducers is that they are orthogonal to space and time, push or pull, applying equally well to most of the entities I describe in this article.

[getify]

Speaking of links, here's more I found in terms of reading:

http://sriku.org/blog/2014/01/25/implementing-csp-channels-using-promises/

http://jlongster.com/Taming-the-Asynchronous-Beast-with-CSP-in-JavaScript

http://swannodette.github.io/2013/07/12/communicating-sequential-processes/

[bergus]

I have done some studies on this now.

It seems to me that CSP channels are exactly like the Promise Buffers of gtor. Instead of yield/write they use the term put, and instead of next/read they use the term take. A channel that is closed seems to relate to our return, but I couldn't figure out a throw equivalent. I guess many people don't see much distinction between them. So much for "Communicating" - they just use streams.

Next, "Processes". Those are much like AIEFEs - asynchronous immediately executed function expressions. While they do run, they put and take values to and from arbitrary channels, typically awaiting the result. They often run forever, only sometimes they're stopped when one their dedicated channels is closed. While in Clojure this is done via tail recursion, JS implementations mostly use while (true) loops. An example:

var inp = new Channel(),
    out = new Channel();
(async function map() {
    while (true) {
        let v = await inp.next();
        let r = …(v);
        await out.yield(r);
    }
}());
// use inp and out channels also within some other processes that provide or consume values

Now, what about "Sequential"? This is imo the biggest difference. While the Promise Iterator next returns a promise that will resolve only when something has been written, and the Promise Generator yield returns a promise that will resolve only when something has been read, they are still asynchronous. A CSP however does block on every put and take call. Multiple processes might call a method on the same channel "simultaneously", but no process can call a method multiple times in a row - without another process performing the dual action in between. The two processed are therefore implicitly synchronized, they "meet" at the time when one supplies a value and the other obtains it. I'm not sure how strictly this requirement is enforced in the current JS implementations of CSP. Only in some of them puts and takes seem to demand the use of yield. As ES7 async/await syntax is not yet available, they use generators for processes which are "spawned" via a go function that takes care of the blocking. On top of this blocking there are also asynchronous combinators, e.g. for racing between two channels, but I haven't looked deeply into them yet.

After all, Promise Streams as described in GTOR (with iterators and generators) seem to model Actors better than CSP, as they have a builtin message queue that can be filled by the same sender. Of course, they are not as powerful as actors yet, the biggest lack I can think of is the missing syntax for receiving messages on an "instance". The asynchronous generator async function* syntax is well suited for sending messages, though.

In the end, I've tried to model the shakespeare example with CSP:

let shakespeare = new Channel();
(async function() {
    for (let title of ["Hamlet", "Macbeth", "Othello"])
        for (let quote of await getQuotes(title))
            await shakespeare.put(quote);
    shakespeare.close();
}());
(async function() {
    var length = 0;
    while (!shakespeare.isClosed)
        length += await shakespeare.take();
    console.log(length);
}());

[kriskowal]

@bergus Foremost, thanks for taking the time to compare the approaches. This is informative.

In GToR I propose the shape of an "asynchronous generator", which would have both await and yield, and return an asynchronous iterator, meaning a readable stream, like a channel.

Aside, it didn’t escape my notice that "next" as a method name may as well have been "yield". The next(write:Value) -> promise<read:Value> method corresponds exactly with the yield Promise<write:Value> :read:Value keyword

Promises can proxy for instances. Q implements "invoke", "get", "call", etc so that messages of these kinds can be pipelined to a remote object. See https://github.com/kriskowal/q-connection

See also the friam (friday morning meeting) group where the Actor people and the Promise people are ironing these distinctions out in the stratosphere https://groups.google.com/forum/#!forum/friam

[bergus]

In GToR I propose the shape of an "asynchronous generator", which would have both await and yield, and return an asynchronous iterator, meaning a readable stream, like a channel.

Yes, I've seen that. However, a channel is also a writable stream, and writable from all "processes" that have a reference to it. Those "asynchronous generators" (async function*) return an asynchronous iterator which is only written to from the generator body using yields.

I get the correspondence between yield and next, though I think that in an async environment typically no values are passed back from the consumer to the producer - except for flow control. I'd more clearly distinguish between them by their types (regardless of the names):

asyncGenerator.yield :: writevalue -> Promise ()
asyncIterator.next   :: () -> Promise readvalue

[kriskowal]

@bergus Agree on all points.

[bergus]

…but maybe I've been indoctrinated by my reading on CSP, and there is a use case of passing values back? When writing the post, I had thought of using an asynchronous generator, but it didn't exactly fit. It's possible to use it in a mix of the styles however:

async function *shakespeare() {
    while (true)
        var quotes = await getQuotes(function.next); // meta keywords ftw :-)
        await yield quotes;
    }
}
var reader = shakespeare();
(async function() {
    let totalLength = 0;
    for (title of ["Hamlet", "Macbeth", "Othello"]) {
        let quotes = await reader.next(title);
        for (let quote of quotes) {
            totalLength += quote.length;
        }
    }
    console.log(totalLength);
}());

Meh, after having written it I can't say I like it. Can anyone come up with a better idea?

[getify]

Definitely check out @jhusain's stuff.. he has a proposal to TC39 for async function*.

https://github.com/jhusain/asyncgenerator

[kriskowal]

@getify We (@domenic and I) were earlier debating whether @jhusain and I had arrived at the same conclusion. I am still foggy on whether there is a distinction between observables and async iterator / readable streams. @jhusain's observables focus on pushing and streams focus on pulling. I would ask, does @jhusain envision observables the way I describe them in GToR, where a fast producer and a slow consumer drop intermediate time-series data rather than buffer and pressurize. There may also be a difference in even vs odd (meaning, does the producer or the consumer move first).

[kriskowal]

Continuing from that thought…

With synchronous generators, the consumer moves first, by calling next() when it wants the producer to resume.

With asynchronous generators, I could see a strong argument for doing the same or different. If they were the same, the async generator would not move until the iterator gets called. Alternately, the generator and iterator could move independently, so the generator would move immediately to the first yield and the producer would move immediately to the first next() call, and the value of the first yield would be resolved with the first argument to next(arg). I do not know which strategy is considered "even" or "odd", nor off-hand who coined the terms, but there’s a link buried in one of these issues.

[kriskowal]

…If I had to guess, the synchronous generator would be called "odd" and the asynchronous generator and iterator that start in parallel would be called "even". I cannot conceive of a reason that would rule out the usefulness of either approach, which implies to me that we might need both.

[jhusain]

Hey Kris,

I wouldn't mind chatting with you about this over hang out. Might be higher bandwidth, and then we can write down the result of our conversation. A chat is long overdue.

[kriskowal]

I plan to do an open office hours hangout on March 22 at 14:00 PST. There is some chance I can make some time this weekend, barring Super ∏ Day observances.

[bergus]

@getify: Thanks for that link, an interesting proposal. However, I don't think the asynchronous generator functions as proposed there match those from GToR. In GToR, they return an asynchronous iterable. This approach seems to be detailed in @zenparsing's proposal, describing an AsyncIterator interface. In contrast, @jhusain's asynchronous generator functions do return Observables, i.e. objects with an @@observer method that will drive another generator. Admittedly, these are a bit unclear to me, as they also return another generator which - as far as I see - can interfere arbitrarily with the driven generator.

[kriskowal]

Ah, yes. Observables are more like the asyn analog of an Iterable (Enumerable in the land in which they were born), whereas an Async Iterator is the async analog of an iterator. Generators return iterators, so it would be idiomatic for Async generators to return async iterators. The distinction is that an "Observable" would be able to replay. I would argue that the async generator function itself is the appropriate model for an observable.

[jhusain]

Let's chat then. send me details at jhusain@gmail.com