NuGet / NuGetGallery

NuGet Gallery is a package repository that powers https://www.nuget.org. Use this repo for reporting NuGet.org issues.
https://www.nuget.org/
Apache License 2.0
1.52k stars 643 forks source link

[Feature]: failed to extract some platform-specific .nupkg file #10022

Closed cyanial closed 1 week ago

cyanial commented 1 week ago

Related Problem

https://www.nuget.org/packages/MongoDB.Libmongocrypt

When NuGetForUnity extract that .nupkg file, it extract two libmongocrypt.dylib on macos, one from content and one from contentFiles.

I can manually delete one of these duplicated dylib and it works.

Now i want to know how to fix this. I've read the code about extracting .nupkg. But i can not know how to fix, or it will cause other lib dont work if i change the code.

The Elevator Pitch

Fix duplicated .nupkg extract in some lib (MongoDB.libmongocrypt) . I want to fix this but don't know how to do.

Additional Context and Details

Multiple plugins with the same name 'libmongocrypt' (found at 'Assets/Packages/MongoDB.Libmongocrypt.1.8.2/Sources/libmongocrypt.dylib' and 'Assets/Packages/MongoDB.Libmongocrypt.1.8.2/content/libmongocrypt.dylib'). That means one or more plugins are set to be compatible with Editor. Only one plugin at the time can be used by Editor.

截屏2024-06-14 16 53 14

cyanial commented 1 week ago

tree view of mongodb.libmongocrypt.1.9.0.nupkg

.
├── License.txt
├── MongoDB.Libmongocrypt.nuspec
├── [Content_Types].xml
├── _rels
├── build
│   └── MongoDB.Libmongocrypt.targets
├── content
│   ├── libmongocrypt.dylib
│   ├── libmongocrypt.so
│   └── mongocrypt.dll
├── contentFiles
│   └── any
│       ├── netstandard2.0
│       │   ├── libmongocrypt.dylib
│       │   ├── libmongocrypt.so
│       │   └── mongocrypt.dll
│       └── netstandard2.1
│           ├── libmongocrypt.dylib
│           ├── libmongocrypt.so
│           └── mongocrypt.dll
├── lib
│   ├── netstandard2.0
│   │   └── MongoDB.Libmongocrypt.dll
│   └── netstandard2.1
│       └── MongoDB.Libmongocrypt.dll
├── package
│   └── services
│       └── metadata
│           └── core-properties
│               └── d8d42d1e094a432abe3b150908d6778c.psmdcp
└── runtimes
    ├── linux
    │   └── native
    │       └── libmongocrypt.so
    ├── osx
    │   └── native
    │       └── libmongocrypt.dylib
    └── win
        └── native
            └── mongocrypt.dll

22 directories, 19 files

tree view after NuGetForUnity extract

├── License.txt
├── License.txt.meta
├── MongoDB.Libmongocrypt.nuspec
├── MongoDB.Libmongocrypt.nuspec.meta
├── Sources
│   ├── libmongocrypt.dylib
│   ├── libmongocrypt.dylib.meta
│   ├── libmongocrypt.so
│   ├── libmongocrypt.so.meta
│   ├── mongocrypt.dll
│   └── mongocrypt.dll.meta
├── Sources.meta
├── content
│   ├── libmongocrypt.dylib
│   ├── libmongocrypt.dylib.meta
│   ├── libmongocrypt.so
│   ├── libmongocrypt.so.meta
│   ├── mongocrypt.dll
│   └── mongocrypt.dll.meta
├── content.meta
├── lib
│   ├── netstandard2.1
│   │   ├── MongoDB.Libmongocrypt.dll
│   │   └── MongoDB.Libmongocrypt.dll.meta
│   └── netstandard2.1.meta
└── lib.meta

5 directories, 22 files
cyanial commented 1 week ago

code of extract .nupkg file

// unzip the package
using (var zip = ZipFile.OpenRead(cachedPackagePath))
{
    var libs = new Dictionary<string, List<ZipArchiveEntry>>();
    var csFiles = new Dictionary<string, List<ZipArchiveEntry>>();
    var anyFiles = new Dictionary<string, List<ZipArchiveEntry>>();

    foreach (var entry in zip.Entries)
    {
        var entryFullName = entry.FullName;

        if (PluginRegistry.Instance.HandleFileExtraction(package, entry, baseDirectory))
        {
            continue;
        }

        if (PackageContentManager.ShouldSkipUnpackingOnPath(entryFullName))
        {
            continue;
        }

        // we don't want to unpack all lib folders and then delete all but one; rather we will decide the best
        // target framework before unpacking, but first we need to collect all lib entries from zip
        const string libDirectoryName = "lib/";
        if (entryFullName.StartsWith(libDirectoryName, StringComparison.Ordinal))
        {
            var frameworkStartIndex = libDirectoryName.Length;
            var secondSlashIndex = entryFullName.IndexOf('/', frameworkStartIndex);
            if (secondSlashIndex == -1)
            {
                // a file inside lib folder -> we keep it. This is to support packages that have no framework dependent sub directories.
                PackageContentManager.ExtractPackageEntry(entry, baseDirectory);
                continue;
            }

            var framework = entryFullName.Substring(libDirectoryName.Length, secondSlashIndex - frameworkStartIndex);
            FillFrameworkZipEntries(libs, framework, entry);

            continue;
        }

        // in case this is a source code package, we want to collect all its entries that have 'cs' or 'any' set as language
        // and their frameworks so we can get the best framework later
        const string contentFilesDirectoryName = "contentFiles/";
        if (entryFullName.StartsWith(contentFilesDirectoryName, StringComparison.Ordinal))
        {
            // Folder structure for source code packages:
            // └─<packageID>.<packageVersion> (not counted here since entries start with next subfolder)
            //   └─<contentFiles>
            //     └─<any> (language)
            //       └─<any> (framework)
            //         └─<sources>
            // In order to create shorter paths, we aim to make a structure like this:
            // └─<packageID>.<packageVersion>
            //   └─Sources
            //      └─<sources>
            var directoriesSplit = entryFullName.Split('/');
            if (directoriesSplit.Length >= 4)
            {
                var language = directoriesSplit[1];
                var framework = directoriesSplit[2];

                if (language.Equals("cs", StringComparison.Ordinal))
                {
                    FillFrameworkZipEntries(csFiles, framework, entry);
                }

                if (language.Equals("any", StringComparison.Ordinal))
                {
                    FillFrameworkZipEntries(anyFiles, framework, entry);
                }
            }

            // if the entry is in content files we want to skip unpacking it right now anyway
            continue;
        }

        PackageContentManager.ExtractPackageEntry(entry, baseDirectory);
    }

    // go through all lib zip entries and find the best target framework, then unpack it
    if (libs.Count > 0)
    {
        var bestFrameworkMatch = TargetFrameworkResolver.TryGetBestTargetFramework(
            libs,
            packageConfig.TargetFramework,
            framework => framework.Key);
        if (bestFrameworkMatch.Value != null)
        {
            NugetLogger.LogVerbose(
                "Selecting target framework directory '{0}' as best match for the package {1}",
                bestFrameworkMatch.Key,
                package);
            foreach (var entry in bestFrameworkMatch.Value)
            {
                PackageContentManager.ExtractPackageEntry(entry, baseDirectory);
            }
        }
        else
        {
            Debug.LogWarningFormat(
                "Couldn't find a library folder with a supported target-framework for the package {0}",
                package);
        }
    }

    // go through all content files' frameworks and figure the best target network, prioritizing 'cs' over 'any' language
    if (csFiles.Count > 0)
    {
        TryExtractBestFrameworkSources(csFiles, baseDirectory, package, packageConfig);
    }
    else if (anyFiles.Count > 0)
    {
        TryExtractBestFrameworkSources(anyFiles, baseDirectory, package, packageConfig);
    }
}
cyanial commented 1 week ago

sorry, issue on wrong place