Question about Asynchronous Observation

[josh]

Hey @jhusain,

I'm really excited by your Async Generator proposal.

I was going to ask more about back pressure and buffering. Then I saw you posted some more info on "Asynchronous Observation" in f53771d0ddfc47f70c7332f01dfac5b1996cd981. Still had a question.

The contract of waiting on next()'s return promise makes sense, but I'm wondering how its enforced? Will all Observables follow this pattern or will Async Observables be a specific subtype? If so how would you tell?

Looking at your current standard library observables, most don't implement buffering just yet. And without it, basic sequential iteration involving async doesn't quite work.

var i = 0;
for (var name on Observable.interval(10)) {
  i++;
  await Observable.timeout(1000).done();
  if (i >= 3) {
    console.log(i);
    return;
  }
}

I seems a little error prone if the user is required to know if the observable is async or not. The final example makes reference to stockNames.buffer(), but I'm not sure where Observable#buffer comes from.

Maybe the observer decorate could implement a buffering strategy on next() such that the current observable implementations just all work?

Thanks!

[jhusain]

Great questions. There's no way to tell by looking at an Observable whether or not it is an Asynchronous Observable (meaning it waits on promises) because it is a pattern implemented by a structural type.

The recommended approach is to convert the structural type to a nominal type ASAP. This allows you to use instanceof comparisons, but more importantly to get access to combinators that respect backpressure.

I haven't yet written the AsyncObservable nominal type yet, only the Observable nominal type. That may be the source of some of the confusion. Neither of these nominal types are proposed for standardization, but instead would be expected to be provided in an Underscore-like library.

I will check in the AsyncObservable nominal type eventually. It will work like this:

var values = [1,2,3]; var asyncObservable = AsyncObservable.from( (async function*() { for (let x of values) { await yield x; } }());

// The asynchronous observable has all of the same methods as observable, except they respect backpessure.

Once you have a nominal type for observable and a nominal type for asynchronous observable, you can add adapter methods to convert the former to the latter. You can convert a regular Observable, say one backed by a DOM event, to an AsyncObservable by calling the buffer method on Observable.

mouseMoves.buffer() // this returns the Nominal AsyncObservable type

Buffer and drop are two different strategies you can use to convert an observable to an asynchronous observable. Buffer add any items that arrive while waiting on the consumer to a buffer. Drop simply throws away items that arrive while waiting on the consumer to handle the value that was delivered to them.

This stuff will be clearer when I check it in, but hopefully I've given you an idea of how this will work.

Dictated using voice recognition. Please forgive the typos.

On Dec 3, 2014, at 12:18 AM, Joshua Peek notifications@github.com wrote:

Hey @jhusain,

I'm really excited by your Async Generator proposal.

I was going to ask more about back pressure and buffering. Then I saw you posted some more info on "Asynchronous Observation" in f53771d. Still had a question.

The contract of waiting on next()'s return promise makes sense, but I'm wondering how its enforced? Will all Observables follow this pattern or will Async Observables be a specific subtype? If so how would you tell?

Looking at your current standard library observables, most don't implement buffering just yet. And without it, basic sequential iteration involving async doesn't quite work.

var i = 0; for (var name on Observable.interval(10)) { i++; await Observable.timeout(1000).done(); if (i >= 3) { console.log(i); return; } } I seems a little error prone if the user is required to know if the observable is async or not. The final example makes reference to stockNames.buffer(), but I'm not sure where Observable#buffer comes from.

Maybe the observer decorate could implement a buffering strategy on next() such that the current observable implementations just all work?

Thanks!

— Reply to this email directly or view it on GitHub.

[josh]

Great! Thanks for the response.

[josh]

Buffer and drop are two different strategies you can use to convert an observable to an asynchronous observable. Buffer add any items that arrive while waiting on the consumer to a buffer. Drop simply throws away items that arrive while waiting on the consumer to handle the value that was delivered to them.

I've definitely seen those before, but wouldn't you always want to use .buffer() in an for on? It seems like the only way to preserve the await semantics. Like maybe that could be the default in the transpiled output?

[jhusain]

The problem with automatically adding a buffer operation is that...

a) this method would need to be added to the standard b) The observable may already respect back pressure in which case you may end up double buffering or unnecessarily buffering. c) buffering can lead to unbounded memory growth in theory. This detail is not something that a programming language should hide from you. In practice you will want control over how buffering is applied. Should it have a maximum size (probably)? Should it throw if the maximum size is exceeded or drop? These are questions that the developer needs to think about and handle explicitly.

J

Dictated using voice recognition. Please forgive the typos.

On Dec 4, 2014, at 10:25 AM, Joshua Peek notifications@github.com wrote:

Buffer and drop are two different strategies you can use to convert an observable to an asynchronous observable. Buffer add any items that arrive while waiting on the consumer to a buffer. Drop simply throws away items that arrive while waiting on the consumer to handle the value that was delivered to them.

I've definitely seen those before, but wouldn't you always want to use .buffer() in an for on? It seems like the only way to preserve the await semantics. Like maybe that could be the default in the transpiled output?

— Reply to this email directly or view it on GitHub.