All file enumerations methods fail with System.IO.IOException: 'An unexpected network error occurred' when enumerating large network foler structures

[HEIC-to-JPEG-Dev]

Description

Enumeration fails when using any of the file enumeration methods such as File.IO.GetFiles, EnumerateFiles, EnumerateFileSystemEntries or DirectoryInfo.... on large network structures.

x86 and x64 debug in VS, running against a Synology NAS using SMB3 on a local network - folder has about 1.2 million files in 60k folders.

Reproduction Steps

EnumerationOptions enumerationOptions = new() { BufferSize = 65535, IgnoreInaccessible = true, RecurseSubdirectories = true, AttributesToSkip = FileAttributes.Device }; var files = Directory.GetFiles(testPath, "*", enumerationOptions);

Expected behavior

It should be able to enumerate the files - It works fine on the same file structure for SSD and HDD, just not over the network.

Actual behavior

After about 5 minutes, all enumeration methods fail with System.IO.IOException: 'An unexpected network error occurred'

Regression?

Error is on .net 6,7,8,9

Known Workarounds

Enumerate folder by folder via recursion in code

Configuration

.Net 9 Windows 11 23H2 x64 Issue is the same from a Windows 10 test machine

Other information

No response

[dotnet-policy-service[bot]]

Tagging subscribers to this area: @dotnet/area-system-io See info in area-owners.md if you want to be subscribed.

[HEIC-to-JPEG-Dev]

Also, the enumerations don't utililize the network bandwidth - a Simple File Explorer "copy" from the same Network Device uses about 550 Mbps, where-as .Net Enumeration uses 14 Mbps - utilization of the NAS (network, CPU, Memeory) is <15% on enumeration

[huoyaoyuan]

What about enumerating SMB share from another Windows machine? The SMB connection is handled by the operating system, not .NET. It's possible that third party implementation is not fully compliant.

[HEIC-to-JPEG-Dev]

What about enumerating SMB share from another Windows machine? The SMB connection is handled by the operating system, not .NET. It's possible that third party implementation is not fully compliant.

As stated in the issue - this has also been tested from a windows 10 machine. Also to note, File Explorer has no issues, but possibly has a different implementation.

If this is not a .Net issue, then fine, I'll continue with the work-around

[huoyaoyuan]

As stated in the issue - this has also been tested from a windows 10 machine.

I mean sharing from another Windows machine, not enumerating from.

Also to note, File Explorer has no issues, but possibly has a different implementation.

I'm more interested about the original failure. What does the "after 5 minutes" count from? Does an enumeration itself costs over 5 minutes?

[HEIC-to-JPEG-Dev]

As stated in the issue - this has also been tested from a windows 10 machine.

I mean sharing from another Windows machine, not enumerating from.

Also to note, File Explorer has no issues, but possibly has a different implementation.

I'm more interested about the original failure. What does the "after 5 minutes" count from? Does an enumeration itself costs over 5 minutes?

Unfortunately I don't have anothe rmachine with the disk space required (4.5 TB) to test that. I'll try knocking it down to SMB 2 or changing settings and see what happens.

Interestingly, I just ran another test while File Explorer was copying network files and when the Directory.GetFiles failed in VS, it also caused the File Explorer copy to fail with a "Too Many files" error (allows me to carry on after a retry).

The 5 minute count is from pressing "run in debug" in VS to it failing, this number changes depending on what my network is doing

[huoyaoyuan]

If different sources are erroring at the same moment, it's probably caused by underlying network connection. Can you consistently reproduce the "5 minute" timing?

[HEIC-to-JPEG-Dev]

I can consistently reproduce the error - just let it enumerate. The timing is dependant on what else I'm doing on the network It would appear that there is some sort of buffer that gets filled and runs out of memory/capacity - but that's internal to the .Net or network code. File Explorer can complete the task, .Net cannot (although .Net does seem to cause File Explorer to error when it errors)