Async functions

[timstableford]

I only noticed your Issue https://github.com/ceifa/wasmoon/issues/13 awhile creating this so that's a positive coincidence.

I originally tried to use Emscripten's Asyncify functionality but couldn't figure out a way without modifying Lua code. So what this does is if a callback returns a promise it calls lua_yieldk with a reference to the promise so it bubbles up to lua_resume. I did a lot of Googling about how to handle arbitrary references thinking there'd be an equivalent to napi_ref from Node but I stumbled on an Issue in their repo where they say to just keep a registry similar to how functions work. I also added a wrapper function around lua_resume that awaits promises and continues until the function is done yielding.

As a bonus to the above I've allowed pushValue to store JS values as references if the option is passed and getValue maps the references back to their JS objects.

Entirely tangentially to the promise handling I also added code to get current memory use and set a maximum. The custom allocator is currently always passed to lua_newstate, I'm happy to not do so if you'd prefer.

Please also let me know if you see a better way of doing Promise handling.

The next items you're likely to see from me are:

• Setting a hook to force a yield every X instructions, along with a timeout to abort if execution takes too long.
• A way of setting the environment variables.
• Checking that arrays can be passed back to JS as arrays rather than objects and doing a bit of conversion if they aren't.
• Allowing a library user to specify custom outputs for stdout and stderr.
• Some tests and a bit of investigation to see if any of Lua's standard libraries could break anything like over-writing the GC meta-table's or such and maybe adding a sandbox mode, not sure though there. The solution there might be exposing the functions for loading individual libraries and making sure it's possible for users of this library disable or replace global functions.

[ceifa]

Great work!

With this solution to promises, how can I call async functions without needing to create a new thread from JS? For example:

    const engine = await getEngine()

    engine.global.set('asyncCallback', async (input) => {
        return Promise.resolve(input * 2)
    })

    const value = engine.doString(`
        local resume = coroutine.wrap(function()
            print(asyncCallback(15)) -- ""
        end)
        print(resume(co)) -- 15
        print(resume(co)) -- ""
        -- coroutine is dead now
    `)

I expected to log a "30" somewhere, but it didn't.

[timstableford]

Afaik it's only possible to yield from a thread rather than the main state.

I'm not certain what's happening in your example. I'll do a bit of investigation. We could modify the doString and doFile helpers to create the new thread automatically perhaps?

Try and call it like this https://github.com/ceifa/wasmoon/blob/297e95ccdf8c53f59f4a56c376db574f6069e260/test/engine.test.js#L237

[timstableford]

I suspect with coroutine.wrap it doesn't know when the promise is resolved

[ceifa]

I was thinking to create an API like that:

local promise = asyncCallback(15)
promise:then(print) -- The same behaviour of JS then
promise:catch(error) -- The same behaviour of JS catch
local result = promise:await() -- Will only work inside a thread(coroutine), basically will yield while the promise is pending and then return the result value

What do you think?

[ceifa]

Answering your question, I don't think creating a new thread automatically it's a good approach because as a client I will probably expect the code to run on the main thread.

[timstableford]

I'm having trouble figuring out the flow of how it would work. If you only had a :then I think the Lua program would just exit early before the then resolves unless it's await as per your example. With catch it may be better to use Lua's built-in try/catch by letting users wrap the async calls in a pcall? I very much see where you're coming from by trying to expose JS functionality directly to it for flexibility I'm just not sure of the underlying details without somehow blocking the program ending like Node does while there's pending timers.

[timstableford]

Answering your question, I don't think creating a new thread automatically it's a good approach because as a client I will probably expect the code to run on the main thread.

I understand what you mean, that could make things rather confusing. I think it's the only way it will allow yielding but I can use lua_xmove to move the results from the thread to the calling state?

[ceifa]

I agree and I don't have concrete answers to your questions. :/ I'm going to merge because you did a great start for promises support and we can improve it later. Thanks again for contributing this! 👍

[ceifa]

@timstableford I think that there is nothing to do about the early return, this actually happens if you bind setTimeout, for example. I implemented a next(then is a keyword) function to see how it would be, take a look at #15.

Something we can do is create some API like await engine.global.waitUnresolvedPromises().