Adding MS netcore 3.1.0 packages fails with duplicate key exceptions (in zip processing)

[coleman-c]

Describe the bug

Adding MS netcore 3.1 packages fails with System.ArgumentException: Item has already been added. This is due to 'packageIcon.png' being in the package twice (see screenshot below)

I'm not sure where the issue lies, but it is successfully indexed by nuget.org so thought I'd raise it here first.

To Reproduce

Steps to reproduce the behavior:

1. Using Nuget.Server 3.4.1 create a new nuget server (.NET 4.6.1 was used) as detailed here: https://docs.microsoft.com/en-us/nuget/hosting-packages/nuget-server
2. Download microsoft.extensions.configuration.3.1.0.nupkg from https://www.nuget.org/packages/Microsoft.Extensions.Configuration/
3. Upload it to the nuget server by placing in the ~/Packages directory
4. See error below:

   Verbose: File system changed. File: microsoft.extensions.configuration.3.1.0.nupkg - Change: Changed
   Info: Start adding packages from drop folder.
   Error: An exception occurred while handling a file system event: System.ArgumentException: Item has already been added. Key in dictionary: 'packageIcon.png'  Key being added: 'packageIcon.png'
   at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
   at System.Collections.Hashtable.Add(Object key, Object value)
   at System.Collections.Specialized.OrderedDictionary.Add(Object key, Object value)
   at MS.Internal.IO.Zip.ZipIOCentralDirectoryBlock.ParseRecord(BinaryReader reader, Int64 centralDirectoryOffset, Int32 centralDirectoryCount, Int64 expectedCentralDirectorySize)
   at MS.Internal.IO.Zip.ZipIOCentralDirectoryBlock.SeekableLoad(ZipIOBlockManager blockManager)
   at MS.Internal.IO.Zip.ZipIOBlockManager.get_CentralDirectoryBlock()
   at MS.Internal.IO.Zip.ZipArchive.GetFiles()
   at System.IO.Packaging.ZipPackage.ContentTypeHelper..ctor(ZipArchive zipArchive, IgnoredItemHelper ignoredItemHelper)
   at System.IO.Packaging.ZipPackage..ctor(Stream s, FileMode mode, FileAccess access, Boolean streaming)
   at System.IO.Packaging.Package.Open(Stream stream, FileMode packageMode, FileAccess packageAccess, Boolean streaming)
   at NuGet.PackageHelper.GetManifestStream(Stream packageStream)
   at NuGet.OptimizedZipPackage.EnsureManifest()
   at NuGet.OptimizedZipPackage..ctor(IFileSystem fileSystem, String packagePath, IFileSystem expandedFileSystem)
   at NuGet.Server.Core.PackageFactory.Open(String fullPackagePath)
   at NuGet.Server.Core.Infrastructure.ServerPackageRepository.AddPackagesFromDropFolderWithoutLocking()
   at NuGet.Server.Core.Infrastructure.ServerPackageRepository.<AddPackagesFromDropFolderAsync>d__40.MoveNext()
   --- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Server.Core.Infrastructure.ServerPackageRepository.<FileSystemChangedAsync>d__54.MoveNext()

   Expected behavior

   The package is indexed

Screenshots

Renaming the nupkg as a zip and inspecting illustrates the issue:

Additional context

The same appears true with a variety of other 3.1 packages I've looked at.

[skofman1]

Sorry for the slow response. @pranavkm , looks like an issue with the package. What do you think? The package icon isn't even configured in the nuspec, so it appears with the default icon on nuget.org.

@nkolev92 , how come the client manages to install this package? Shouldn't it fail during unzip?

[pranavkm]

We're aware of the issue and it's being fixed in our upcoming patch release: https://github.com/aspnet/Extensions/issues/2744. There's also a issue for the NuGet client to flag duplicate items: https://github.com/NuGet/Home/issues/6941

[nkolev92]

@nkolev92 , how come the client manages to install this package? Shouldn't it fail during unzip? @skofman1

We don't seem to validate that. We have a tracking issue for it on the extraction side. https://github.com/NuGet/Home/issues/8875

@pranavkm correctly linked the issue for the pack side of things.

[daconglee]

iisexpress.exe Error: 0 : System.ArgumentException: 已添加项。字典中的关键字:“packageIcon.png”所添加的关键字:“packageIcon.png” 在 System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add) 在 System.Collections.Hashtable.Add(Object key, Object value) 在 System.Collections.Specialized.OrderedDictionary.Add(Object key, Object value) 在 MS.Internal.IO.Zip.ZipIOCentralDirectoryBlock.ParseRecord(BinaryReader reader, Int64 centralDirectoryOffset, Int32 centralDirectoryCount, Int64 expectedCentralDirectorySize) 在 MS.Internal.IO.Zip.ZipIOCentralDirectoryBlock.SeekableLoad(ZipIOBlockManager blockManager) 在 MS.Internal.IO.Zip.ZipIOBlockManager.get_CentralDirectoryBlock() 在 MS.Internal.IO.Zip.ZipArchive.GetFiles() 在 System.IO.Packaging.ZipPackage.ContentTypeHelper..ctor(ZipArchive zipArchive, IgnoredItemHelper ignoredItemHelper) 在 System.IO.Packaging.ZipPackage..ctor(Stream s, FileMode mode, FileAccess access, Boolean streaming) 在 System.IO.Packaging.Package.Open(Stream stream, FileMode packageMode, FileAccess packageAccess, Boolean streaming) 在 System.IO.Packaging.Package.Open(Stream stream) 在 NuGet.PackageHelper.GetManifestStream(Stream packageStream) 在 NuGet.OptimizedZipPackage.EnsureManifest() 在 NuGet.OptimizedZipPackage..ctor(IFileSystem fileSystem, String packagePath, IFileSystem expandedFileSystem) 在 NuGet.Server.Core.PackageFactory.Open(String fullPackagePath) 在 NuGet.Server.Core.Infrastructure.ServerPackageRepository.AddPackagesFromDropFolderWithoutLocking() 在 NuGet.Server.Core.Infrastructure.ServerPackageRepository.d__48.MoveNext()

[skofman1]

nuget.org should block packages with duplicate files. When the client adds logic to block those, we will take the new bits.