Tokio runtimes and async

[bendk]

I've been thinking about the current async_runtime support and wondering how useful it is:

• If all you want to do is use tokio types and not manage any threads, it seems better to use versions of those types that are executor-agnostic like async-mutex and async-timer
• If you want tokio to manage a threadpool, then async_compat isn't a great solution for this since it doesn't allow you to customize the tokio Runtime.

What if instead of using async-compat, we allowed users to specify a function that returns a type that derefs to tokio::Runtime and let them build their own runtime? Something like:

use once_cell::sync::Lazy;
use tokio::runtime::{Builder,Runtime};

fn main_runtime() -> &'static Lazy<Runtime> {
    static RUNTIME: Lazy<Runtime> = Lazy::new(|| {
         Builder::new_multi_thread()
             .worker_threads(4)
             .thread_name("my-custom-name")
             .thread_stack_size(3 * 1024 * 1024)
             .build()
             .unwrap()
    });
    &RUNTIME
}

#[uniffi::export(tokio_runtime = main_runtime)]
pub async fn use_shared_resource(options: SharedResourceOptions) -> Result<(), AsyncError> {
    ...
}

Then inside the scaffolding function, we do what async-compat does and enter the runtime before polling the future.

I'm not sure that this is how it should work though. @jplatte @Hywan are you currently using UniFFI async with tokio? If so, how does it work?

[jplatte]

Yes we are using this, and we are patching async-compat to be a fork that exposes the runtime.. 😄

However, the main reason we are accessing the runtime directly is that we still use a bunch of block_on, i.e. functions that should be async at the FFI boundary, but aren't. I think the main thing we need from tokio is the blocking thread pool and timers, and we don't care that much about async tasks actually running in parallel. I agree though that it's a bit limiting, so if you have a use case for it, I'm totally open to changing things. I'm not super enthusiatic about the design you sketched above, but that's purely a vibe thing and I'll think about what bothers me about it / how it could be done differently.

[bendk]

Yeah, I'm not sure if I love the ergonomics of my proposal.

If the main point is thread pools another option would be for UniFFI doesn't do any wrapping and requires that users manually call Runtime::spawn/Runtime::spawn_blocking. One thing I like about that is that it makes it more explicit what's happening on which executor.

[Hywan]

I agree that the current async_runtime implementation is limited but it has the merit to work.

What I don't feel clear with your proposal is: how is it supposed to work? It's focusing on defining a Runtime, which is a specific type of tokio in this case. Each async lib comes with its runtime definition. I believe the correct abstraction level is Future, as async-compat does (I'm not saying we should stick with it, but I reckon the approach is correct —though too strict/limiteed for now).

We would need a way to express we want to wrap the outgoing Future inside another Future that can be build in some way. Something like:

#[uniffi::export(async_wraps_with = future_wrapper)]
pub async fn foo(…) -> … { }

fn future_wrapper<F, T>(future: F) -> F
where F: Future<Output = T>>
{
    // … fetch `Runtime` from somewhere
    // build a new `Future` that uses `Runtime`
    // i.e. it simply is a “custom re-implementation” of `async_compat::Compat`.
}

Thoughts?

Edit: I believe that with this proposed design, it's even possible to simply write #[uniffi::export(async_wraps_with = async_compat::Compat::new)] in the case of the basic experience (i.e. if you don't need to access the Runtime of tokio from async_compat).

[jplatte]

@Hywan I don't think we should be over-abstracting things. There isn't really a use case for this outside of tokio compatibility, is there? I think there's a discussion to be had on whether tokio compat should be something UniFFI bothers with at all, but if it does I don't see much merit to abstracting it such that it can theoretically handle things other than tokio, that nobody actually needs.

[Hywan]

My proposal allows to choose the runtime and to configure the runtime you want per function. It can be a nice feature to have actually :-).

[jplatte]

Well, other runtimes don't need explicit compatibility code like tokio does. The only other major runtime, async-std, uses a lazily-initialized global runtime so it does not need this.

[mhammond]

My proposal allows to choose the runtime and to configure the runtime you want per function.

We've a vaguely-defined use-case that would like to choose the runtime dynamically at runtime (roughly, a kind of adaptor that would either want to use an async stack supplied by the foreign code or supplied by a Rust implementation, depending on the environment the code finds itself in) - @bendk is working through making that more concrete though...