convenience of await syntax and multiple results

[ousado]

The current proposed design requires different await wrapper functions for different types of Promises, Tasks, etc., as well as additional parenthesis, both of which is quite clumsy. It also requires to bind parameters for direct CPS-style functions, which in turn requires an additional allocation. It doesn't account for multiple results (from e.g. a CPS-style callback), which also requires an additional allocation plus an additional function call as a workaround.

With direct support for an async keyword or meta and explicit (extensible) compiler-support for arbitrary types, none of these would be an issue.

[nadako]

The current proposed design requires different await wrapper functions for different types of Promises, Tasks, etc.

Some code to support any kind of task abstraction would be required anyway, so I don't see an issue here. And compared to some "extension" thingy required to tell the compiler how to await, I find it more simple and straightforward.

as well as additional parenthesis

True, but it's quite natural if you think about it as an actual function (and it can be an extension method too, if desired). Note that await is not the only application of suspendable functions. Other good examples are yield(...) for generators, channel.send(...) for Go-like stuff. And the fact that user can have their own custom suspension points allow for nice and concise domain-specific coroutines.

It also requires to bind parameters for direct CPS-style functions, which in turn requires an additional allocation.

Not true:

typedef Continuation<T> = {
    function resume(value:T):Void;
    function error(error:Dynamic):Void;
}

class Main {
    // generated await function
    static inline function await<T>(fn:(Dynamic->T->Void)->Void, cont:Continuation<T>) {
        fn(function(err, res) if (err != null) cont.error(err) else cont.resume(res));
    }

    // some cps function
    static function readFile(path:String, cb:Dynamic->String->Void) {
        cb(null, "hi");
    }

    // state machine mock
    static var stateMachine = {
        resume: function(r) {
            trace(r);
        },
        error: function(e) {
            throw e;
        }
    }

    static function main() {
        // generated usage
        await(readFile.bind("test.txt"), stateMachine);
    }
}

generates:

Main.main = function() {
    var cont = Main.stateMachine;
    Main.readFile("test.txt",function(err,res) {
        if(err != null) {
            cont.error(err);
        } else {
            cont.resume(res);
        }
    });
};

As also noted in the current README, explicit .bind could be avoided by a simple macro that transforms e.g. await(readFile(path)) await(readFile(path, _)) into _await(readFile.bind(path))

It doesn't account for multiple results (from e.g. a CPS-style callback), which also requires an additional allocation plus an additional function call as a workaround.

I'd rather leave this quite specific case for later optimization, but I don't immediately see a problem of allowing multiple return values for coroutines without additional allocations. It would require changing the interaction protocol a bit, which is not yet set in stone anyway.

With direct support for an async keyword or meta

I'll mention it again that coroutines are not only useful for async/await kind of programming style and there are more to them, some of which are already mentioned in the current README and I guess waiting for more examples. Some of them are: generators, Go-like concurrency where basically everything is a coroutine, animation/scenario mini-DSLs, super-highlevel business logic that can be asynchronous under the hood, but only as an implementation detail.

and explicit (extensible) compiler-support for arbitrary types

Which is something everybody keeps talking about but never proposes an actual design draft. I imagine it being either too complex (as in: ocaml plugins or super-complex declarative configuration) or too slow to scale well (macros).

[ousado]

True, but it's quite natural if you think about it as an actual function (and it can be an extension method too, if desired). Note that await is not the only application of suspendable functions. Other good examples are yield(...) for generators, channel.send(...) for Go-like stuff. And the fact that user can have their own custom suspension points allow for nice and concise domain-specific coroutines.

I'm all for flexibility - I'm against unnecessary verbosity.

Not true:

bind itself requires an additional allocation.

I'll mention it again that coroutines are not only useful for async/await kind of programming style ...

Yes that's right, but that doesn't mean that we shouldn't have explicit, convenient support for await. This is not an either or situation. If one size doesn't fit it all (and it doesn't) then why not design something that's flexible enough - for instance by having both suspend and async.

Which is something everybody keeps talking about but never proposes an actual design draft.I imagine it being either too complex (as in: ocaml plugins or super-complex declarative configuration) or too slow to scale well (macros).

I implemented it in macros so far, but it would be quite easy to generate a template from haxe code that is then applied in ocaml directly.

[nadako]

bind itself requires an additional allocation.

as shown in the example right above - not always, because of immediately invoked function optimization.

[ousado]

I really don't get why you're against supporting CPS-style (and more generally type-driven implementation selection) out of the box - without macros and additional function calls, syntactically or otherwise. We have everything that's required, we could provide very elegant and convenient support, and that wouldn't even prevent us from supporting your use-case, too.

[nadako]

What I like about my approach is that it's really generic and straightforward to both implement (in the compiler) and use (by simply writing suspending functions) and not tied to a single particular problem domain (awaiting asynchronous operations), unlike "classic" async/await.

Anyway, feel free to propose another design that we could actually compare to see all advantages and disadvantages of both. Until then, we should just start prototyping the state-machine transformation in the compiler, since that part will be more or less the same, not depending on the final syntax.

[nadako]

Oh and to answer your question, I see several approaches to await a CPS function and I'm really not sure which I would prefer.

For a promise-like API I see no difference really between await load(...), await(load(...)) and load(...).await(). If anything, the last two explicitly hint that load(...) call returns some value that can be awaited.

For a continuation-passing API there's no perfect syntatic solution. load(arg, ret -> ...) is clearly what we want to avoid, but how? I don't like await load(arg) because the last(?) callback arg becomes implicit, await load(arg, _) is a bit better, but still requires knowledge about the underscore transforming mechanics and I'm not sure how does it play with other underscores (e.g. in extractors or custom macros). OTOH await load.bind(arg) would be easier to understand since we clearly just passing a (partially applied) function to some construct that will call it with a callback. But in this case this is no better than doing await(load.bind(arg)). In both cases one could have their own cheap macro transformation to use whatever syntax they like.

So I really don't see why should we hard-code specific use cases into the compiler for the questionable convinence instead of having a generic and straightforward coroutine feature.

[ousado]

First of all, I don't think a different design is necessary, what's missing is a type-driven selection mechanism for async/await, and perhaps that concept could be extended to other use cases, too.

Regarding your concerns with the await load(arg, _) syntax, supporting extractors and guards will be a tough exercise anyway (I'm not sure it should even be supported), I don't think whatever choices we make here would add much to that complexity, and macros need to be aware of language constructs anyway.

It's true that this syntax doesn't cover all possible CPS API flavors (e.g. what to do with separate callbacks for results and errors), that's definitely something to think about.

Regarding "questionable convenience": Let's say a regular user of thx on nodejs who also happens to use some library that uses js.Promise watches a tink_web-related talk from the Haxe summit and decides to check it out:

Please compare:

suspend function doStuff() {
    var a = awaitThxPromise(thxFunc(a1, a2));
    var b = awaitTinkPromise(tinkWebFunc(b1, b2));
    var c = awaitCPS(nodeFunc.bind(c1, c2));
    var d = awaitJSPromise(someFunc(a, b, c));
    return createThxPromiseWith(d);
}

with

suspend function doStuff() : thx.Promise {
    var a = await thxFunc(a1, a2);
    var b = await tinkWebFunc(b1, b2);
    var c = await nodeFunc(c1, c2, _));
    var d = await someFunc(a, b, c);
    return d;  // automatically wrapped in a thx.Promise by the thx.Promise provider 
}

Please note that

• I'm all for generic coroutine support, and think that's good idea
• I think we should hard-code as little behavior as possible, and make the mechanism as flexible as possible.

Regarding implementing the selection functionality in macros - that's relatively complicated and certainly expensive, because this feature obviously depends on the availability of type information. First class support during typing would be a much better option.

An implementation provider would have to implement an interface that roughly looks like this:

interface AsyncProvider {
    function onEntry(..):Expr;    // handles the entry-point, e.g. creating and returning a Promise
    function onAwait(..):Expr;    // handles an await call
    function onError(..):Expr;    // hooks into the error handling mechanism
    // ...  (maybe more depending on the granularity of our support for different error handling strategies)
    function onReturn(..):Expr; // handles the final return, e.g. resolving the initially returned Promise
}

As already mentioned, the implementation selection is separate from the state machine, it maybe would have to be able to inject entry points into the state machine, perhaps deal with the optimization you describe, but for the most part all it has to do is selecting those awaitSpecificHandler() functions (or generate the inlined code directly) based on type, and do a similar thing on entry and exit.

[nadako]

Oops, I forgot about this discussion :)

suspend function doStuff() {
    var a = awaitThxPromise(thxFunc(a1, a2));
    var b = awaitTinkPromise(tinkWebFunc(b1, b2));
    var c = awaitCPS(nodeFunc.bind(c1, c2));
    var d = awaitJSPromise(someFunc(a, b, c));
    return createThxPromiseWith(d);
}

IMO this should be solved by supporting method overloading, so we can use await in all cases. Without overloading support, a static extension can be used to provide similar .await() methods to all of them.

Regarding building a promise, it's more like return createThxPromiseWith(() -> { ... method body... }), as shown in the current README, and the function itself would not be suspend, because it's supposed to return a promise, not suspend the calling coroutine.

So, it should be more like this:

function doStuff() return createThxPromise(() -> {
    var a = await(thxFunc(a1, a2));
    var b = await(tinkWebFunc(b1, b2));
    var c = await(nodeFunc.bind(c1, c2));
    var d = await(someFunc(a, b, c));
    return d;
});

---

Anyway, if we're talking about a convenience layer on top of the proposed design, I'm not against it, but I also think it's quite narrow-focused, since it only covers async/await case.

[ousado]

The async/await case alone is quite complex already. There's still a lot to think about: multiple returns, error handling, timeouts, parallel/throttled/sequential scheduling, perhaps even some notion of resource management. All of that requires attention and some of it benefits from dedicated compiler support. The automatic wrapping based on the return type as demonstrated above is just one example for that. You might think about it as narrow-focused, but it's not exactly a small design space to cover if we want to come up with actually strong support.

Coroutines are a general mechanism, yeah, but that doesn't mean they're the end all be all of everything that can be implemented on top, and I don't see for what reason the discussion should artificially be limited at that point.

[Simn]

It's normal to go from general to specific. As long as we make sure we don't make decisions that block more specific "extensions" from following up, I don't really see the problem with this approach.

However, I agree with not calling it "narrow-focused" because the subject itself is far from narrow.