Possibility of having exclusions in GetDirectories / GetFiles

[RandomSheeple]

I keep constantly running into exceptions for trying to access various folders and files that for real but varying reasons (even over time) just cannot or should not be enumerated or accessed in any way. Or if we take the poor user defense, filesysteminfo objects I would not want to include. And the thing is internally in my software I have exclusion lists, just no way that I am aware of, of passing them to AlphaFS enumeration functions.

As there are constant exceptions in cases where logically (not physically in the file system) I already know these will happen, these can be considered needless exception handling raising, et cetera. Since I do capture errors in a central DB it's easy to see repeating folder and file names that are blocked usually by accessdenied errors but sometimes others. There are mysterious errors with MacOS encrypted files, and other nice bits and pieces like HP VAX copied over files from the 1990s. But I digress. Regardless of the type of error, if it is repeating and reliable, I would wish to skip these entirely.

Typical examples are windows system excluded or blocked paths such as [x:]\$RECYCLE.BIN, somehow system level blocked folders in %TEMP%, [X:]\RECOVERY, [X]:\system volume information. For data centers at least EMC and NetApp have various folders on their boxen (the names of which change by firmware or something), wherein you may NOT even look, and when (if ever) you have access rights, changing time stamps by even touching things will horribly break stuff like multi-data-center internal replication caches. Or so I am told. And when scanning large drive sets one usually tends to have the program run with really high privileges.

So for these reasons, and I am sure over time other reasons may emerge, the ability to have AlphaFS skip even just explicitly specified folder / file names, would be really great. By explicit I mean not wildcarding or anything, just really specific explicit paths / names. On the negative side, I am not sure it is even possible on the OS level or C++ level. I mean even logically how do you enumerate that which you do not want to enumerate.... I'm guessing the support would have to be somewhere really deep in the OS where the AlphaFS components are running.

[Yomodo]

Perhaps you can try using the Directory.EnumerateFileSystemEntryInfos method and experiment with the DirectoryEnumerationFilters which uses callbacks/predicates. In the callbacks you can use all the code needed to do custom filtering and/or Exceptions and continue enumerating.

When run elevated, it will also help to enable the Alphaleonis.Win32.Security.Privilege.Backup privilege which will give you access to folders which are normally inaccessible.

C#: EnumerateFileSystemEntryInfos using DirectoryEnumerationFilters.

PowerShell: EnumerateFileSystemEntryInfos using DirectoryEnumerationFilters.

Personally I prefer using EnumerateDirectories/EnumerateFiles instead of GetDirectories/GetFiles. With the latter out of memory exceptions can surface fast; the GetXxx methods build up an entire array in memory, whereas EnumeratieXxx starts returning results the moment they are available.

[Yomodo]

@RandomSheeple Is this of any help, can I close the issue?

[astrohart]

Hi @Yomodo the link in your post above, which points to "C#: EnumerateFileSystemEntryInfos using DirectoryEnumerationFilters", at URL: https://github.com/alphaleonis/AlphaFS/blob/develop/AlphaFS.UnitTest/AlphaFS%20FileSystemEntryInfo%20Class/AlphaFS_Directory.EnumerateFileSystemEntryInfos_EnumerateFiles_UsingDirectoryEnumerationFilters.cs, is broken. Please fix it. Thank you.

[astrohart]

Hi @Yomodo does your suggested use of DirectoryEnumerationFilters solve the issue of when you are using Directory.EnumerateDirectories, say, and during the enumeration an UnauthorizedAccessException occurs? because, otherwise, I am unsure how to tell the code to just skip that folder when enumerating; right now, when I make the call below, and then it hits that folder that has the UnauthorizedAccessException, everything just crashes and burns. I want to indeed just skip the folder that is having the issue instead.

using Alphaleonis.Win32.Filesystem;

var folders = Directory.EnumerateDirectories(
    Environment.ExpandEnvironmentVariables(
           @"%USERPROFILE%\Documents"
    ),
    "*",
    SearchOption.AllDirectories
);

Thanks for any tips.

This is similar to the issue raised over here on Stack Overflow: https://stackoverflow.com/questions/13130052/directoryinfo-enumeratefiles-causes-unauthorizedaccessexception-and-other