Suggestion: optimize NuGet.Packaging.PackageBuilder.AddFiles()

Suggestion: NuGet.Packaging.PackageBuilder.AddFiles() can avoid call to ResolveSearchPattern when items in PackageFiles are all absolute paths.

Issue: When hundreds of files are being added (via PackageFiles task property), the tight loop in AddFiles() becomes a choke point for some filesystems (such as VirtioFS used in docker-mac fileshare), and throws intermittent errors like:

 System.IO.DirectoryNotFoundException: Could not find a part of the path '/runtime/src/installer/pkg/sfx/Microsoft.NETCore.App'.
    at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
    at System.IO.Enumeration.FileSystemEnumerator`1.Init()
    at System.IO.Enumeration.FileSystemEnumerableFactory.UserFiles(String directory, String expression, EnumerationOptions options)
    at System.IO.Directory.InternalEnumeratePaths(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
    at System.IO.Directory.GetFiles(String path, String searchPattern, EnumerationOptions enumerationOptions)
    at NuGet.Common.PathResolver.PerformWildcardSearch(String basePath, String searchPath, Boolean includeEmptyDirectories, String& normalizedBasePath)
    at NuGet.Packaging.PackageBuilder.ResolveSearchPattern(String basePath, String searchPath, String targetPath, Boolean includeEmptyDirectories)
    at NuGet.Packaging.PackageBuilder.AddFiles(String basePath, String source, String destination, String exclude)
    at NuGet.Packaging.PackageBuilder.PopulateFiles(String basePath, IEnumerable`1 files)
    at NuGet.Commands.MSBuildProjectFactory.CreateBuilder(String basePath, NuGetVersion version, String suffix, Boolean buildIfNeeded, PackageBuilder builder)
    at NuGet.Commands.PackCommandRunner.BuildFromProjectFile(String path)
    at NuGet.Build.Tasks.Pack.PackTask.Execute() (TaskId:86)
Done executing task "PackTask" -- FAILED. (TaskId:86)

(40% of builds failed in mounted setup on macOS; all of them with same PackTask callstack)

Note this is only a suggestion to reduce inodes usages (the LINQ query causes n^m lookups which seems a bit too much for what the domain requirement is). ENOENT (No such file or directory) errors on VirtioFS is reported to be fixed in upcoming macOS 15 and latest docker (I'm on Sonoma 14.5, the current stable release). Feel free to close this if refactoring doesn't warrant. 🙂

cc @akoeplinger, this is related to https://github.com/dotnet/runtime/pull/98597#issuecomment-1953365183.

NuGet / Home

Suggestion: optimize NuGet.Packaging.PackageBuilder.AddFiles() #13572