Nuget Restore/Install of packages.config from local feed with non-normalized versions fails to locate nupkg

ghost commented 5 years ago

Details about Problem

NuGet product used (NuGet.exe | VS UI | Package Manager Console | dotnet.exe): Nuget.exe

NuGet version (x.x.x.xxx): 5.2.0.6090 (reproduces back to at least 3.6)

dotnet.exe --version (if appropriate): N/A

VS version (if appropriate): N/A

OS version (i.e. win10 v1607 (14393.321)): Server 2016 v1607 (14393.3181). This is not specific to a particular version of Windows.

Worked before? If so, with which NuGet version: No

Detailed repro steps so we can see the same problem

Locate or create a NuGet package whose version can be normalized (eg. Owin 1.0 can normalize to Owin 1.0.0). Legacy packages with 4-number versioning and a trailing 0 also apply (Foo 1.2.3.0 normalize to Foo 1.2.3). Azure DevOps (ADO) may also have some packages with extraneous 0's such as Package.1.2.01 instead of the normalized Package.1.2.1.
Generate the .nupkg.sha512 file, or use the corresponding file from the user's global-cache if the package was downloaded via NuGet
Place the .nupkg, .nupkg.sha512, and .nuspec files into a directory (eg. C:\Test) following the V3 cache format (eg. C:\Test\Owin\1.0.0\). If the package was downloaded via NuGet, using the global-cache directory or copying its contents will suffice.
De-normalize the nupkg and nupkg.sha512 file names (eg. make them Owin.1.0.nupkg and Owin.1.0.nupkg.sha512)
Generate a packages.config requesting the test package. It does not matter if the version is normalized or not (ie. requesting 1.0 and 1.0.0 will both reproduce this issue)
Add the test directory from step 3 as a NuGet source. Preferably, remove all other package sources from nuget.config (or generate a new nuget.config).
Run nuget restore -OutputDirectory -NoCache

The NoCache flag is to avoid finding the package in the user's global-cache. Omit the flag if using the global cache as the test directory in step 3.

Expected behavior: nuget locates and restores the package. Actual behavior: nuget does not locate the nupkg and does not restore the package.

Other suggested things

Notes

nuget will successfully restore the package if the source is not using the hierarchical folder structure (eg. C:\Test\Owin.1.0.nupkg instead of C:\Test\Owin\1.0.0\Owin.1.0.nupkg) and will do so regardless of mismatch between nupkg name and the version requested in packages.config
When restoring a package nuget will place the nupkg on disk as the remote sends it but will always normalize the nupkg in the user's global cache. For example, If ADO or nuget.org send Owin.1.0.nupkg, nuget will place Owin.1.0.nupkg in the destination but place Owin.1.0.0.nupkg and Owin.1.0.0.nupkg.sha512 in %HOMEDIR%.nuget\packages
Our use case is that we take the package downloaded by nuget and propagate it to other machines to avoid sending requests to ADO packaging feeds.

Per discussion with @rrelyea, there are four potential paths to fixing this:

Fix the packages when we propagate them to have normalized versions in their file names. 2a. Fix nuget to normalize nupkg files in the destination location 2b. Make nuget check for non-normalized nupkgs when traversing a hierarchical source
If nuget.org always returns normalized nupkgs, fix ADO to also return normalized nupkgs
Effectively do not propagate these packages to other machines and have nuget always send requests to package feeds.

Verbose Logs

C:\Foo>nuget.exe restore packages.config -OutputDirectory C:\Foo -Verbosity detailed -NoCache NuGet Version: 5.2.0.6090 Restoring NuGet package OWin.1.0.0. WARNING: Unable to find version '1.0.0' of package 'OWin'. file://C:/Test: Package 'OWin.1.0.0' is not found on source 'file://C:/Test'.

WARNING: Unable to find version '1.0.0' of package 'OWin'. file://C:/Test: Package 'OWin.1.0.0' is not found on source 'file://C:/Test'.

Errors in packages.config projects Unable to find version '1.0.0' of package 'OWin'. file://C:/Test: Package 'OWin.1.0.0' is not found on source 'file://C:/Test'.

NuGet Config files used: C:\Users\joshumay\AppData\Roaming\NuGet\NuGet.Config C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

Feeds used: file://C:/Test

Sample Project

Test package layout: TestPackageSource.zip Test packages.config: TestPackagesConfig.zip

anangaur commented 5 years ago

Related normalization issue to not normalize at pack time: https://github.com/NuGet/Home/issues/3050

emgarten commented 4 years ago

NuGet hierarchical feeds (v3 folder format) require nupkg file names to be normalized. This has been a requirement since it was introduced in NuGet 3.0.0 and then later back ported to 2.x.

Nupkgs with non-normalized file names can still be used from a local folder, you just can't mix the hierarchical feed format (feeds with .sha512 files) with packages in the older local package folder format.

NuGet add command should be used to add packages to a hierarchical feed to ensure that the file names are correct. It also handles creating the hash file.

StingyJack commented 4 years ago

What is an example of a "normalized file name" in this context?

NuGet / Home