Async callbacks in Lua?

earlysun030201 commented 2 months ago

The function created using the 'create sync_function' function will block Lua execution. Is there a wrapper that can implement the following code? lua:

settimeout(function()    -- created by create_async_function   settimeout(f: LuaFunction, t: u64)
    print("foo")
end, 1000)

print("global")

result:

global
foo

earlysun030201 commented 2 months ago

The current result I have is that timeouts always block the execution of subsequent code, and only execute subsequent code after the timeout is completed. result after 2000 milliseconds:

foo
global

earlysun030201 commented 2 months ago

resolved

    let set_timeout = lua.create_async_function(|lua, (callback, time): (LuaFunction, u64)| async move {
        let callback_key: RegistryKey = lua.create_registry_value(callback)?;

        tokio::spawn(async move {
            tokio::time::sleep(Duration::from_millis(time)).await;

            if let Ok(callback) = lua.registry_value::<LuaFunction>(&callback_key) {
                let _ = callback.call::<()>(());
            }

            let _ = lua.remove_registry_value(callback_key);
        });

        Ok(())
    })?;

khvzak commented 2 months ago

it does not block, it awaits set_timeout when you call it from Lua.

using the spawn - one of the solutions. another one:

tokio::join!(
        lua.load(mlua::chunk! {
            $set_timeout(function() print("foo") end, 1000)
        })
        .exec_async(),
        lua.load(mlua::chunk! {
            print("global")
        })
        .exec_async(),
    );

earlysun030201 commented 2 months ago

it does not block, it awaits set_timeout when you call it from Lua.

using the spawn - one of the solutions. another one:
tokio::join!(
        lua.load(mlua::chunk! {
            $set_timeout(function() print("foo") end, 1000)
        })
        .exec_async(),
        lua.load(mlua::chunk! {
            print("global")
        })
        .exec_async(),
    );

Yes, it doesn't block the rust process, but in lua it waits for the asynchronous function to finish running, and what I'm trying to do is to return the value via an asynchronous callback in the same piece of lua code instead of waiting for it to finish and then returning the value. This will greatly increase efficiency when I send multiple asynchronous http requests at the same time in the same piece of lua code :p

khvzak commented 2 months ago

Yeah, tokio::spawn is the way to go. On the other note, you don't need to create registry values, you can pass LuaFunction directly to async task (since it Send+Sync+'static):

    let set_timeout = lua
        .create_async_function(|_, (callback, time): (LuaFunction, u64)| async move {
            tokio::spawn(async move {
                tokio::time::sleep(Duration::from_millis(time)).await;
                let _ = callback.call::<()>(());
            });
            Ok(())
        })?;

earlysun030201 commented 2 months ago

Yeah, tokio::spawn is the way to go. On the other note, you don't need to create registry values, you can pass LuaFunction directly to async task (since it Send+Sync+'static):
    let set_timeout = lua
        .create_async_function(|_, (callback, time): (LuaFunction, u64)| async move {
            tokio::spawn(async move {
                tokio::time::sleep(Duration::from_millis(time)).await;
                let _ = callback.call::<()>(());
            });
            Ok(())
        })?;

pub fn build_set_timeout(lua: &Lua) -> LuaFunction {
    let set_timeout = lua.create_async_function(|_, (callback, time): (LuaFunction, u64)| async move {
        tokio::spawn(async move {
            tokio::time::sleep(std::time::Duration::from_millis(time)).await;
            let _ = callback.call::<()>(());
        });
        Ok(())
    }).map_err(|e| e.into_lua_err()).unwrap();

    return set_timeout;
}

Can't seem to make it... Perhaps the problem is caused by ownership management, when I stop using registry value, the following error message is output:

Lua instance is destroyed

The configuration I use is as follows:

[dependencies.mlua]
version = "0.10.0-beta.2"
features = ["luajit", "async", "send", "serialize"]
optional = true

OS: Macos Sequoia Beta 24A5331b CPU: Apple M1 Pro More error outputs:

Lua instance is destroyed
stack backtrace:
   0: rust_begin_unwind
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panicking.rs:652:5
   1: core::panicking::panic_fmt
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/panicking.rs:72:14
   2: core::panicking::panic_display
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/panicking.rs:263:5
   3: core::option::expect_failed
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/option.rs:1994:5
   4: core::option::Option<T>::expect
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/option.rs:895:21
   5: mlua::state::WeakLua::lock
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mlua-0.10.0-beta.2/src/state.rs:1866:23
   6: mlua::function::Function::call
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mlua-0.10.0-beta.2/src/function.rs:103:19
   7: kanade_fofa_lib::lua::modules::common::build_set_timeout::{{closure}}::{{closure}}::{{closure}}
             at ./src/lua/modules/common.rs:7:21
   8: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/core.rs:331:17
   9: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/loom/std/unsafe_cell.rs:16:9
  10: tokio::runtime::task::core::Core<T,S>::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/core.rs:320:13
  11: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:500:19
  12: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/panic/unwind_safe.rs:272:9
  13: std::panicking::try::do_call
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panicking.rs:559:40
  14: ___rust_try
  15: std::panicking::try
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panicking.rs:523:19
  16: std::panic::catch_unwind
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panic.rs:149:14
  17: tokio::runtime::task::harness::poll_future
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:488:18
  18: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:209:27
  19: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:154:15
  20: tokio::runtime::task::raw::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/raw.rs:271:5
  21: tokio::runtime::task::raw::RawTask::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/raw.rs:201:18
  22: tokio::runtime::task::LocalNotified<S>::run
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/mod.rs:436:9
  23: tokio::runtime::scheduler::multi_thread::worker::Context::run_task::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/worker.rs:598:13
  24: tokio::runtime::coop::with_budget
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/coop.rs:107:5
  25: tokio::runtime::coop::budget
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/coop.rs:73:5
  26: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/worker.rs:597:9
  27: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/worker.rs:548:24
  28: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/worker.rs:513:21
  29: tokio::runtime::context::scoped::Scoped<T>::set
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/context/scoped.rs:40:9
  30: tokio::runtime::context::set_scheduler::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/context.rs:180:26
  31: std::thread::local::LocalKey<T>::try_with
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/thread/local.rs:286:12
  32: std::thread::local::LocalKey<T>::with
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/thread/local.rs:262:9
  33: tokio::runtime::context::set_scheduler
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/context.rs:180:9
  34: tokio::runtime::scheduler::multi_thread::worker::run::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/worker.rs:508:9
  35: tokio::runtime::context::runtime::enter_runtime
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/context/runtime.rs:65:16
  36: tokio::runtime::scheduler::multi_thread::worker::run
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/worker.rs:500:5
  37: tokio::runtime::scheduler::multi_thread::worker::Launch::launch::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/worker.rs:466:45
  38: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/blocking/task.rs:42:21
  39: tokio::runtime::task::core::Core<T,S>::poll::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/core.rs:331:17
  40: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/loom/std/unsafe_cell.rs:16:9
  41: tokio::runtime::task::core::Core<T,S>::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/core.rs:320:13
  42: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:500:19
  43: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/panic/unwind_safe.rs:272:9
  44: std::panicking::try::do_call
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panicking.rs:559:40
  45: ___rust_try
  46: std::panicking::try
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panicking.rs:523:19
  47: std::panic::catch_unwind
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panic.rs:149:14
  48: tokio::runtime::task::harness::poll_future
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:488:18
  49: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:209:27
  50: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/harness.rs:154:15
  51: tokio::runtime::task::raw::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/raw.rs:271:5
  52: tokio::runtime::task::raw::RawTask::poll
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/raw.rs:201:18
  53: tokio::runtime::task::UnownedTask<S>::run
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/task/mod.rs:473:9
  54: tokio::runtime::blocking::pool::Task::run
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/blocking/pool.rs:160:9
  55: tokio::runtime::blocking::pool::Inner::run
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/blocking/pool.rs:518:17
  56: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /Users/weed/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/blocking/pool.rs:476:13

earlysun030201 commented 2 months ago

I have to explicitly move the ownership of lua to the code block created by spawn in order to be able to call the callback. in theory having the move keyword is enough isn't it?

pub fn build_set_timeout(lua: &Lua) -> LuaFunction {
    let set_timeout = lua.create_function(move |lua, (callback, time): (LuaFunction, u64)| {
        let lua = lua.clone();

        std::thread::spawn( move ||{
            let _ = lua.clone();
            std::thread::sleep(std::time::Duration::from_millis(time));
            callback.call::<()>(()).ok();
        });

        Ok(())
    }).map_err(|e| e.into_lua_err()).unwrap();

    return set_timeout;
}

output:

khvzak commented 2 months ago

Lua instance is destroyed means that the Lua instance is dropped while other threads are still active. Cloning and moving Lua inside threads is fine. Other possible solutions: wait for all threads to join OR make as Lua static global

mlua-rs / mlua

Async callbacks in Lua? #453