WalkDir error loop

[forensicmatt]

I am having an issue with an infinite loop of error messages when using WalkDir. Using the std::fs::read_dir is able to move past the error.

Here is the code I am using for example with WalkDir:

let walk_iter = WalkDir::new(source_location).into_iter();

for entry_result in walk_iter {
    let entry = match entry_result {
        Ok(entry) => entry,
        Err(e) => {
            eprintln!("{:?}", e);
            continue;
        }
    };

    println!("{}", entry.path().display());
}

Example output:

> .\target\release\examples\iter_dir_walkdir.exe -s D:\Windows\System32\winevt\Logs
D:\Windows\System32\winevt\Logs
D:\Windows\System32\winevt\Logs\Application.evtx
...
D:\Windows\System32\winevt\Logs\Microsoft-Windows-WindowsUpdateClient%4Operational.evtx
D:\Windows\System32\winevt\Logs\Microsoft-Windows-WinINet-Config%4ProxyConfigChanged.evtx
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
Error { depth: 1, inner: Io { path: None, err: Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." } } }
...

Here is an example function I am using to iterate files with fs::read_dir:

fn handle_directory(directory_path: &str) {
    for dir_reader in fs::read_dir(directory_path) {
        for entry_result in dir_reader {
            match entry_result {
                Ok(entry) => {
                    let path = entry.path();
                    if path.is_file() {
                        let path_string = path.into_os_string().into_string().unwrap();
                        println!("{}", &path_string);
                    } else if path.is_dir(){
                        let path_string = path.into_os_string().into_string().unwrap();
                        handle_directory(
                            &path_string
                        );
                    }
                },
                Err(error) => {
                    eprintln!("Error reading {} [{:?}]", directory_path, error);
                    break;
                }
            }
        }
    }
}

With the resulting error being:

> .\target\release\examples\iter_dir_std.exe -s D:\Windows\System32\winevt\Logs
D:\Windows\System32\winevt\Logs\Application.evtx
...
D:\Windows\System32\winevt\Logs\Microsoft-Windows-WindowsUpdateClient%4Operational.evtx
D:\Windows\System32\winevt\Logs\Microsoft-Windows-WinINet-Config%4ProxyConfigChanged.evtx
Error reading D:\Windows\System32\winevt\Logs [Os { code: 1392, kind: Other, message: "The file or directory is corrupted and unreadable." }]

[forensicmatt]

It seems to have something to do with DirList. Is there anything you can think of that might not be allowing this to work as expected? https://github.com/BurntSushi/walkdir/blob/master/src/lib.rs#L612

It seems the idea was indeed to stop iterating after an error, and this seems to be the case. I am having trouble figuring out why this specific instance it continues yielding the error message.

[forensicmatt]

So, I found that a pop is needed here: https://github.com/BurntSushi/walkdir/blob/7c7013259eb9db400b3e5c7bc60330ca08068826/src/lib.rs#L663

Example:

match next {
    None => self.pop(),
    Some(Err(err)) => {
        self.pop();
        return Some(Err(err))
    },
    Some(Ok(dent)) => {
        if let Some(result) = self.handle_entry(dent) {
            return Some(result);
        }
    }
}

Otherwise, it keeps iterating on the same directory that has the error. It fixes my issue. Do you have a thought on why there wouldn't be a pop here or any issues you could see with having a pop here?

[forensicmatt]

The standard library also get stuck in an infinite loop on the error (if I don't add a break on error), which makes me think that this might be a fs::read_dir issue.

This seems kind of similar to https://github.com/rust-lang/rust/issues/50619, which I saw was also referenced in an older issue for walkdir here https://github.com/BurntSushi/walkdir/issues/98. Just for Windows and not Linux.

[forensicmatt]

The file system I am running this on is a mounted file system, so its certainly more of a unique situation.

[BurntSushi]

It sounds like this might be stale at this point? Please feel free to re-open if your explorations yield anything!