async/await and callback support

[christianrondeau]

I was surprised to find no issues nor pull requests to support C#/JS async/await and Task support.

In a perfect world, I could make that transparent and "halt" the calling method (like #192), or use a callback mechanism (which would make scripts much more complex however).

I'd prefer to avoid using return task.Result and friends, so that I don't get into a deadlock eventually.

Before investigating this further, did anyone actually attempt any async/await support in Jint? @sebastienros is this something you'd be interested in supporting, e.g. through the native ECMA-262 async/await (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function)? Any failed or partial attempt?

To avoid any confusion, given this C# code:

public async Task<string> loadSomething(string path) {
  return await File.ReadAllTextAsync(path);
}

I'd consider any of these JavaScript equivalents to be correct (in order of preference):

// Pausing (easiest for script writers):
var x = loadSomething('test.txt');
// Async Function (concise)
var x = await loadSomething('test.txt');
// Promises (well known)
var x;
loadSomething('test.txt').then(function(result) { x = result });;
// Callbacks (worst case)
var x;
loadSomething('test.txt', function(result) { x = result });;

Any ideas or feedback on that is welcome; I'd like to avoid spending any time on this if it would not be merged, or if the effort is too large for what I can afford.

[danbopes]

I would love to see something like what EnCey prototyped implemented. Right now, there are no easy ways to schedule task continuations, and wait for them to all finish asynchronously.

[EnCey]

In case anyone wants to try it: my prototype doesn't work with the async/await keywords (in JavaScript code), but it works with regular JavaScript promises.

The reason is that Jint's await implementation unwraps a promise & throws if it isn't settled, and naturally a promise backed by an async C# Task won't be settled most of the time. The problem doesn't exist with regular promises because the unsettled promise is returned as result of the synchronous execution, at which point my prototype awaits all Tasks and then executes the promise callbacks.

[danbopes]

@EnCey I attempted to tackle this issue on my own, and this was the problem I ran into as well. There isn't some magical way to stop execution in the javascript engine, and asynchronously wait the result. The solution I came up with, unfortunately does use the .GetAwaiter().GetResult() which isn't ideal, but since I've got a thread dedicated to executing the engine, I don't mind it tying up while it's waiting for the result. When I initialize a promise, I pass the task to the promise. When the engine attempts to get the value from a promise that hasn't resolved yet, I make it wait for the task synchronously and then return the result. This still does make it async in nature, because you can fire off multiple tasks in the background, but as soon as the engine needs one of the results, it will wait synchronously for it.

When I attempted to do it the "right way", which involved async ALL the way down, the code became sloppy. All of the .Execute() and .GetValue()'s for everything need to be turned into .ExecuteAsync() and .GetValueAsync(), which complicates things to a big degree. I don't see anyway to "pause" execution, since the .GetValue() is directly trying to get the value of that function, so it's not a simple task to turn one awaiter into an .ExecuteAsync().

[christianrondeau]

It might be better to consider using ValueTask instead of task, but yeah either we have async all the way (simple but doubles everything if we really want to keep a non async version) OR we have a mechanism to save execution state and resume it later (this would be the optimal solution but it's not obvious to implement)

[EnCey]

A third option might be to transpile the JavaScript code first to not use async/await, but regular promises. There's an issue here somewhere that describes using the TypeScript compiler to transform JS code. I've played with that and it did work, allowing me to use async/await in TypeScript code and my prototype to execute the resulting JS code.

One problem here is that without source maps, any error messages are not helpful to users, as they refer to the (messy) generated code. The elephant in the room of course being how (in)efficient it is to run the entire TypeScript compiler with Jint to pre-process code (although once transpiled, the resulting JS code can be cached to reduce that impact).

[lofcz]

This makes another case for implementing source maps support. Tracked here.

[2763071736]

This is my code

            var jintEngine = new Engine();
            jintEngine.SetValue("myAsyncMethod", new Func<Task>(async () =>
            {
                await Task.Delay(1000);
                Debug.Log("myAsyncMethod");
            }));
            jintEngine.SetValue("myAsyncMethod2", new Action(() =>
            {
                Debug.Log("myAsyncMethod2");
            }));
            jintEngine.Execute("async function hello() {await myAsyncMethod();myAsyncMethod2();} hello();");

I expect the output to be

myAsyncMethod
myAsyncMethod2

But await does not seem to wait on a C# function. I got the following wrong result

myAsyncMethod2
myAsyncMethod

I am using v3.0.0-beta-2049

[SGStino]

@2763071736, i believe that's separate from the whole async/await discussion here. It looks to me like the task doesn't get converted to a promise.

Run this in roslynpad:

#r "nuget: Jint, 3.0.0-beta-2049"
using Jint;

var jintEngine = new Engine();
            jintEngine.SetValue("myAsyncMethod", new Func<Task<string>>(async () =>
            {
                await Task.Delay(1000);
                "myAsyncMethod".Dump();
                return "testvalue";
            }));

            jintEngine.SetValue("myAsyncMethod2", new Action<string>((string value) =>
            {
                value.Dump();
                "myAsyncMethod2".Dump();
            }));

jintEngine.Execute("async function hello() {  const test = await myAsyncMethod(); myAsyncMethod2(test); } hello();"); 

await Task.Delay(2000); // give roslynpad time to wait for the delay 

And you'll get the dumps: System.Runtime.CompilerServices.AsyncTaskMethodBuilder+AsyncStateMachineBox[System.String,Program+<>c+<<<Main>$>b__0_0>d] and myAsyncMethod2

followed by myAsyncMethod a bit later.

but the hint here is the AsyncTaskMethodBuilder.

based on this discussion you can use the TryConvert code to wrap the Task.Delay in a promise:

#r "nuget: Jint, 3.0.0-beta-2049"
using Jint;
using Jint.Native;
using Jint.Native.Promise;

var tcs = new TaskCompletionSource<object>();

var jintEngine = new Engine();

jintEngine.SetValue("_resolveFinal", tcs.SetResult);

 Thread.CurrentThread.Dump("starting thread");

jintEngine.SetValue("dump", new Func<object, object>(obj => obj.Dump("jintDump")));
jintEngine.SetValue("myAsyncMethod", new Func<JsValue>(() =>
{
    var (promise, resolve, reject) = jintEngine.RegisterPromise();
    _ = Task.Delay(1000).ContinueWith(_ =>
    {
        Thread.CurrentThread.Dump("continuation thread");
         resolve(JsValue.FromObject(jintEngine, "testvalue"));

        "myAsyncMethod".Dump();
    });

    return promise;
}));

jintEngine.Execute("myAsyncMethod().then(_resolveFinal)");

var final = await tcs.Task;
final.Dump("final");

do mind, you'll be resolving your promise from the timer thread.

That's quickly resolved by using a SynchronizationContext from the nuget Nito.AsyncEx.Context:

#r "nuget: Nito.AsyncEx.Context, 5.1.2"
#r "nuget: Jint, 3.0.0-beta-2049"
using Jint;
using Jint.Native;
using Jint.Native.Promise;
using Nito.AsyncEx;

var tcs = new TaskCompletionSource<object>();

var jintEngine = new Engine();

jintEngine.SetValue("_resolveFinal", tcs.SetResult);

AsyncContext.Run(async () =>
{

    Thread.CurrentThread.Dump("starting thread");
    SynchronizationContext.Current.Dump();

    var scheduler = TaskScheduler.FromCurrentSynchronizationContext();
    jintEngine.SetValue("dump", new Func<object, object>(obj => obj.Dump("jintDump")));
    jintEngine.SetValue("myAsyncMethod", new Func<JsValue>(() =>
    {
        var (promise, resolve, reject) = jintEngine.RegisterPromise();
        _ = Task.Delay(1000).ContinueWith(_ =>
        {
            Thread.CurrentThread.Dump("continuation thread");
            resolve(JsValue.FromObject(jintEngine, "testvalue"));

            "myAsyncMethod".Dump();
        }, scheduler);

        return promise;
    }));

    jintEngine.Execute("myAsyncMethod().then(_resolveFinal)");

    var final = await tcs.Task;
    final.Dump("final");
});

But when trying to use the await keyword version: jintEngine.Execute("async function hello(){ const result = await myAsyncMethod(); _resolveFinal(result); } hello()"); instead of jintEngine.Execute("myAsyncMethod().then(_resolveFinal)");

We get a Jint exception that it wants to unwrap the promise before it was resolved, indicating to me that the await isn't fully implemented as of yet?