uutils / coreutils

Cross-platform Rust rewrite of the GNU coreutils
https://uutils.github.io/
MIT License
17.55k stars 1.26k forks source link

`test_closes_file_descriptors` fails on linux with `Too many open files` #3899

Closed sylvestre closed 2 months ago

sylvestre commented 2 years ago

Seems to be a recent regression


---- test_cp::test_closes_file_descriptors stdout ----
current_directory_resolved: 
run: /home/runner/work/coreutils/coreutils/target/debug/coreutils cp -r --reflink=auto dir_with_10_files/ dir_with_10_files_new/
thread 'test_cp::test_closes_file_descriptors' panicked at 'Command was expected to succeed.
stdout = 
 stderr = cp: '/tmp/.tmptcEL3i/dir_with_10_files/0' -> 'dir_with_10_files_new/0': Too many open files (os error 24)
', tests/common/util.rs:176:9

failures:
    test_cp::test_closes_file_descriptors
jfinkels commented 2 years ago

Assuming dir_with_10_files/0 is visited first (WalkDir says the order is unspecified, but it seems reasonable to assume considering all of the CI builds have the same error), that means there are already too many file descriptors open by the time the directory traversal starts. Maybe another utility is leaking open file descriptors?

sylvestre commented 2 years ago

With procfs, I listed the fd opened in the test.


fd_count = Ok(13)
FDInfo { fd: 0, mode: 0700, target: Socket(33323) } READ | WRITE | EXECUTE
FDInfo { fd: 1, mode: 0700, target: Socket(33325) } READ | WRITE | EXECUTE
FDInfo { fd: 2, mode: 0700, target: Socket(33327) } READ | WRITE | EXECUTE
FDInfo { fd: 3, mode: 00, target: Path("/proc/7459") } (empty)
FDInfo { fd: 4, mode: 0700, target: Socket(33345) } READ | WRITE | EXECUTE
FDInfo { fd: 5, mode: 0700, target: Socket(33346) } READ | WRITE | EXECUTE
FDInfo { fd: 6, mode: 0500, target: Path("/proc/7459/fd") } READ | EXECUTE
FDInfo { fd: 7, mode: 0500, target: Pipe(38780) } READ | EXECUTE
FDInfo { fd: 8, mode: 0500, target: Path("/proc/7459/fd") } READ | EXECUTE
FDInfo { fd: 9, mode: 0500, target: Pipe(38781) } READ | EXECUTE
FDInfo { fd: 112, mode: 0500, target: Pipe(32982) } READ | EXECUTE
FDInfo { fd: 115, mode: 0300, target: Pipe(32983) } WRITE | EXECUTE
current_directory_resolved: 
run: /home/runner/work/coreutils/coreutils/target/debug/coreutils cp -r --reflink=auto dir_with_10_files/ dir_with_10_files_new/
thread 'test_cp::test_closes_file_descriptors' panicked at 'Command was expected to succeed.
stdout = 
 stderr = cp: '/tmp/.tmpwdY3WX/dir_with_10_files/0' -> 'dir_with_10_files_new/0': Too many open files (os error 24)
', tests/common/util.rs:176:9
kralo commented 2 months ago

I could run successfully on my local machine and the test currently passes in CI. Is this worth keeping around?

cakebaker commented 2 months ago

No, I don't think it makes sense to keep this issue open, hence closing it.

@kralo thanks for the hint :)