rmanoka / async-scoped

A scope for async_std and tokio to spawn non-static futures
117 stars 14 forks source link

Why is async_scope blocks some writes to file? #17

Closed chabapok closed 9 months ago

chabapok commented 9 months ago

I have the following minimally reproducible example:

use tokio::fs::{File, OpenOptions};
use tokio::io::AsyncWriteExt;
use tokio::spawn;

fn main() {

    let runtime = tokio::runtime::Builder::new_multi_thread()
        .worker_threads(8)
        .max_blocking_threads(1)
        .enable_all()
        .build()
        .unwrap();

    runtime.block_on(async {
        let t= spawn(async move {
            let mut f: File = OpenOptions::new()
                .create(true)
                .write(true)
                .open("/tmp/foo.txt").await.unwrap();

            let (_r, _outputs) = async_scoped::TokioScope::scope_and_block(|s| {
                s.spawn(async move {
                    println!("----1---");
                    f.write_all(b"asdasd").await.unwrap();
                    println!("----2---");
                    f.write_all(b"asdasd").await.unwrap();
                    println!("----3---");
                });
                ()
            });
        });
        t.await.unwrap();
    });

}

And I use next Cargo.toml dependencies:

[dependencies]
tokio = { version = "1.34.0", features = ["rt", "time", "net", "sync", "rt-multi-thread", "io-util", "fs"] }
async-scoped = { version = "0.8.0", features = ["use-tokio"] }

This code prints ----1--- and ----2---, but not prints ----3---. So, program freezes on second write_all. If i trying to create runtime with .max_blocking_threads(2), program woks ok. Second call to write_all needs an extra thread? Why? It's a bug of the async_scoped crate or not? If I execute write_all without extra spawn in async block, program works ok too.

rmanoka commented 9 months ago

Tokio uses blocking threads for fs operations (as some/most? kernels don't support async fs). We also use one for scope_and_block. So we need two blocking threads.

chabapok commented 9 months ago

Thanks for the info. Sorry for the unnecessary bug report...