File IO hangs after timeout / cancellation

[kmandelbaum]

Version

└── tokio v1.37.0
    └── tokio-macros v2.2.0 (proc-macro)

tokio = { version = "1.37.0", features = ["rt", "rt-multi-thread", "io-std", "time", "io-util", "macros"] }

Platform

Linux hostname 6.8.7-arch1-2 #1 SMP PREEMPT_DYNAMIC Fri, 19 Apr 2024 09:51:31 +0000 x86_64 GNU/Linux

Description

It looks like file reads are not fully cancelled (i.e. block the runtime / hold on to resources) by timeout or select! until the read syscall returns.

I tried this code:

use tokio::io::AsyncReadExt;
#[tokio::main]
async fn main() {
    let mut stdin = tokio::io::stdin();
    println!("Reading...");
    let _ = tokio::time::timeout(
        std::time::Duration::from_secs(1),
        stdin.read_u8()
    ).await;
    println!("Reading done");
}

I expected to see this happen: the program terminates immediately after printing "Reading done".

Instead, this happened: The program prints "Reading...", and then, after one second, "Reading done", then hangs indefinitely until I send something to stdin.

Tried this with stdin and files created by mkfifo, also tried cancellation using timeout and select! between read and tokio::time::sleep, also tried different read* methods with the same result - in all those cases the program hangs in the end until there's input to be read.

When doing the same thing with reading from a TCP socket, the program behaves as expected - terminates immediately after printing "Reading done". When doing an equivalent thing in async-std, the program behaves as expected.

The relevant bits of strace look like this

[pid 181951] 23:38:45 read(0, Reading done
 <unfinished ...>
[pid 181948] 23:38:46 +++ exited with 0 +++
[pid 181953] 23:38:46 +++ exited with 0 +++
[pid 181939] 23:38:46 +++ exited with 0 +++
[pid 181949] 23:38:46 +++ exited with 0 +++
[pid 181952] 23:38:46 +++ exited with 0 +++
...
[pid 181951] 23:38:49 <... read resumed>"\n", 8192) = 1  <- at this point I hit return

I wonder if that's an implementation quirk, and whether this kind of thing matters at all. For my immediate use case it doesn't because I can do std::process::exit and be done with it. That's a workaround though, and there could be a concern with resources leaking because of this. If I run open (for a FIFO) + timeout code in the loop, it seems to be leaking FDs (based on looking into /proc/$pid).

So this one

use tokio::io::AsyncReadExt;
use std::os::fd::AsRawFd;
#[tokio::main]
async fn main() {
    println!("Reading...");
    for i in 0..400 {
        let mut f = tokio::fs::File::open("in").await.unwrap();
        let _ = tokio::time::timeout(
            std::time::Duration::from_micros(1),
            f.read_u8()
        ).await;
        println!("{}", f.as_raw_fd());
    }
    println!("Reading done");
}

accompanied by mkfifo in; cat > in in another terminal also quickly reaches "Reading done", hangs, and shows many FDs open in /proc.

[Darksonn]

Unfortunately, this is a fundamental limitation of blocking IO, which is necessary for ordinary files. There is nothing Tokio can do about it. We have methods for reading from a fifo that work in a non-blocking manner in tokio::net::unix::pipe.

One thing you can do is to shut down your runtime with shutdown_timeout. That will let you exit even if there are hanging blocking IO operations. But it is of course best to avoid them.

[kmandelbaum]

@Darksonn thanks a million!

Yep, I've done some more research, learned a bit more about epoll. tokio::net::unix::pipe does indeed work. I think this should be closed, if it's a well-known limitation. I wonder if tokio::fs could detect if the file is a pipe, and choose the "right" implementation, though it's arguable what is right in that case.