BurntSushi / walkdir

Rust library for walking directories recursively.
The Unlicense
1.21k stars 106 forks source link

IntoIter::count incorrectly returns 0 for \\SomeRemoteServer\C$ #141

Closed andrewbanchich closed 3 years ago

andrewbanchich commented 3 years ago

Example:

    let files = || {
        WalkDir::new("./").into_iter().filter(|path| match path {
            Ok(p) => !p.path().is_dir(),
            Err(_) => true,
        })
    };

    println!("Iter count: {}", files().count());

    let mut n = 0;

    files().for_each(|_entry| {
        n += 1;
    });

    println!("Real count: {}", n);

When I run this on a \\SomeRemoteServer\C$ directory, it will quickly display Iter count: 0 and then eventually Real count: 1530.

When testing by creating a local administrative share \\MyIPAddress\TestShare$, I don't run into this issue.

BurntSushi commented 3 years ago

IntoIter doesn't override the count method, so I don't really see a reason for this and I doubt this is a problem in walkdir. I'd encourage you to debug this further or find a way to make a solid reproducible example. Otherwise, if this is a problem in walkdir, this is unlikely to get fixed.

For example, what is the output of:

    let files = || {
        WalkDir::new("./").into_iter().filter(|path| match path {
            Ok(p) => !p.path().is_dir(),
            Err(_) => true,
        })
    };

    println!("Iter count: {}", files().count());
    println!("Iter count: {}", files().count());
andrewbanchich commented 3 years ago

Interesting... That second count worked! However, I have no idea what could be causing that.

andrewbanchich commented 3 years ago

Wouldn't it likely be related to something in WalkDir since Iterator::count uses Iterator::fold which uses next, which is something implemented by WalkDir?

BurntSushi commented 3 years ago

I don't see how. They are completely independent executions. I'm afraid this will require a solid reproduction or you'll need to debug this bug on your own. If it were me, I would start inserting println calls to isolate the first Windows syscall that returns an unexpected result. And then hopefully you can reconstruct a more minimal example, perhaps without using walkdir at all.

andrewbanchich commented 3 years ago

Thanks! I'll see what I can find.