A simple way to turn callback pattern to coroutine pattern in Lua - Luyu Huang's Tech Blog

[luyuhuang]

https://luyuhuang.tech/2020/09/13/callback-to-coroutine.html

My game project is written by Lua. However, its framework does not provide a coroutine pattern, it uses a simple callback pattern instead. For example, to se...

[nathancahill]

Thanks for this. I used this pattern and it works really well. I ended up nesting coroutines to make some of the functions cleaner.

local function sendCommand(command)
    return coroutine.yield(function(resolve)
        coroutinize(function()
            [do whatever async]
            [do whatever async]
            resolve()
        end)
    end)
end

[luyuhuang]

@nathancahill Thanks for this. I used this pattern and it works really well. I ended up nesting coroutines to make some of the functions cleaner.

Well, this seems like

async function sendCommand(command) {
    await new Promise((resolve) => {
        (async () => {
            [do whatever async]
            [do whatever async]
            resolve();
        })();
    });
}

since sendCommand is yieldable, why not just write those async code directly in sendCommand?

local function sendCommand(command)
    [do whatever async]
    [do whatever async]
    return
end

[nathancahill]

@luyuhuang

I should have filled in a bit more to make it clearer. What I'm trying to do is like this in JS:

async function sendCommand(command) {
    await delay(0.5)
    sendKeyStroke('/')
    await delay(0.5)
    sendKeyStroke(command)
}

In Lua:

local function delay(secs)
    return coroutine.yield(function(resolve)
        hs.timer.doAfter(secs, resolve)
    end)
end

local function sendCommand(command)
    return coroutine.yield(function(resolve)
        coroutinize(function()
            delay(0.5)
            sendKeyStroke('/')
            delay(0.5)
            sendKeyStroke(command)
            resolve()
        end)
    end)
end

coroutinize(function()
    sendCommand('a')
    sendCommand('b')
end)

In JS we don't need further wrappers. But in Lua, without using coroutinize in the sendCommand function, it's not running in a coroutine and delay can't yield. If you see a cleaner way to achieve this, I'd love feedback. I'm just learning Lua.

[luyuhuang]

@nathancahill

Unlike JS, Lua's yield can pass through functions, so you can just:

local function delay(secs)
    return coroutine.yield(function(resolve)
        hs.timer.doAfter(secs, resolve)
    end)
end

local function sendCommand(command)
    delay(0.5)
    sendKeyStroke('/')
    delay(0.5)
    sendKeyStroke(command)
end

coroutinize(function()
    sendCommand('a')
    sendCommand('b')
end)

In fact, you can call coroutine.yield in any function if coroutine.isyieldable returns true.

[nathancahill]

Ah thanks so much. I thought I had tried that before. In any case, it works perfectly. Do you have a tip jar? I'd buy you a coffee.

[luyuhuang]

Ah thanks so much. I thought I had tried that before. In any case, it works perfectly. Do you have a tip jar? I'd buy you a coffee.

My pleasure. if it helped, you can just give me a star. 😊

[JurisBog]

I am really enjoying this! I am a bit curious about the timer functionality with this. I think an interesting idea would be to use it with animation. (while (cond) do ... wait(0) end -> continue next "time"/tick/frame).

Looking at it however, my gut feeling tells me that this implementation increases the stack usage on every "coroutine.yield" and may potentially result in stack overflow. Am I mistaken? Would using return data(exec) fix it? Reference: http://www.lua.org/pil/6.3.html

[luyuhuang]

@JurisBog Looking at it however, my gut feeling tells me that this implementation increases the stack usage on every "coroutine.yield" and may potentially result in stack overflow. Am I mistaken? Would using return data(exec) fix it? Reference: http://www.lua.org/pil/6.3.html

You found an important issue, thank you! I tested and confirmed that it does has such a problem. The follow code causes a stack overflow:

local function co()
    local i = 0
    while true do
        io.stdout:write('\r' .. i)
        i = coroutine.yield(function(resolve)
            resolve(i + 1)
        end)
    end
end

Because exec calls data then data calls exec, it a recursive call. To enable tail-call to fix that problem, we must write return data(exec) and return resolve(i + 1):

function coroutinize(f, ...)
    ...
        if coroutine.status(co) ~= "dead" then
            return data(exec)
    ...
end

local function co()
    local i = 0
    while true do
        io.stdout:write('\r' .. i)
        i = coroutine.yield(function(resolve)
            return resolve(i + 1)
        end)
    end
end

And now the stack looks like this:

stack traceback:
        t.lua:4: in function <t.lua:3>
        (...tail calls...)
        t.lua:13: in local 'coroutinize'
        t.lua:26: in main chunk
        [C]: in ?

[JurisBog]

After dwelling on this for a bit. How do you feel about using iteration instead of recursion? Unfortunately, the parameters would have to go through tbl -> unpack procedure, but I believe it would avoid the issue entirely and would expand possible use cases without unexpected side-effects.

[luyuhuang]

How do you feel about using iteration instead of recursion?

I figure out a different method. To avoid recursion, I instead introduce a await function:

local function await(f)
    local co = coroutine.running()
    local ret
    f(function(...)
        if coroutine.status(co) == "running" then
            ret = table.pack(...)
        else
            return coroutine.resume(co, ...)
        end
    end)
    if ret then
        return table.unpack(ret, 1, ret.n)
    else
        return coroutine.yield()
    end
end

So now we call await instead of coroutine.yield in a coroutine. await calls the function f in the coroutine, instead of yield it and call it from outside the coroutine and that does avoid recursion. We can use it as follow:

local function co()
    local i = 0
    while true do
        io.stdout:write('\r' .. i)
        i = await(function(resolve)
            resolve(i + 1)
        end)
    end
end

coroutine.wrap(co)()

[nathancahill]

I just tried with your new await function and I'm happy to report it works perfectly.

[glepnir]

thanks for this blog. I think create a gist or a repo in github will be better?

[luyuhuang]

@glepnir thanks for this blog. I think create a gist or a repo in github will be better?

sure.