eminence / procfs

Rust library for reading the Linux procfs filesystem
Other
362 stars 106 forks source link

Enable oppportunistic fd counting fast path #234

Closed bobrik closed 1 year ago

bobrik commented 1 year ago

Existing slow path takes ~52000us of CPU time on my laptop to count 100k open files. Compare this to just 13us for the baseline, when no extra files are open by a program:

fd count = 7, elapsed = 13 us
fd count = 100007, elapsed = 51863 us

Adding fastpath from Linux v6.2 makes it fast:

fd count = 4, elapsed = 2 us
fd count = 100004, elapsed = 8 us

This is before taking in account any lock contention effects in the kernel if you try to count files from multiple threads concurrently, which makes the slow path even slower, burning a lot more CPU in the process. See:

I used the following code to benchmark:

use procfs::process::Process;
use std::{fs::File, time::Instant};

fn main() {
    let myself = Process::myself().unwrap();

    let started = Instant::now();
    let count = myself.fd_count().unwrap();
    let elapsed_us = started.elapsed().as_micros();

    eprintln!("fd count = {}, elapsed = {} us", count, elapsed_us);

    let files = (0..100_000)
        .map(|_| File::open("/etc/hosts").unwrap())
        .collect::<Vec<_>>();

    let started = Instant::now();
    let count = myself.fd_count().unwrap();
    let elapsed_us = started.elapsed().as_micros();

    eprintln!("fd count = {}, elapsed = {} us", count, elapsed_us);

    drop(files);
}

See also: https://github.com/prometheus/procfs/pull/486

eminence commented 1 year ago

Looks nice, thanks. I just realized that we don't have any tests for fd_count()... would you mind adding a small commit that calls fd_count -- probably best in test_all() or test_proc_fds()

bobrik commented 1 year ago

Sure, I added a basic test to make sure that the count increments and decrements as expected. It needs to run single threaded to avoid picking up fds from other threads. Let me know if you had something else in mind.

eminence commented 1 year ago

That's perfect, thank you!