Paket should verify nupkg in cache if has same id and content (added maybe from different sources)

[agross]

There are packages with different contents depending on the package source. One such example is Pester which is available on both nuget.org and powershellgallery.com but has a different structure. The nuget.org version has the code inside tools, whereas the PSG one has the code inside the root directory.

During paket restore Paket should verify that the package pulled from the local NuGet cache was downloaded from the same source that is specified in paket.lock.

[forki]

First of all: wtf are these people doing?

Second: how?

[agross]

The nuget.org use case with tools might be to have the PowerShell code not interfere with nuget.exe/VS integration.

The PSG use case is very likely to have to code in the root to make importing the module easier.

[Stift]

I think both packages are used by different package managers originally.

[agross]

Second: how?

On restore Paket knows the expected source URL from the lockfile. Upon reading the JSON from the cache, you can compare it to the SourceUrl.

paket.lock:

NUGET
  remote: https://www.powershellgallery.com/api/v2
  specs:
    Pester (3.4.0)

Pester.3.4.0.something.json:

{
   "Dependencies":[

   ],
   "PackageName":"Pester",
   "SourceUrl":"https://www.powershellgallery.com/api/v2",
   "Unlisted":false,
   "DownloadUrl":"https://www.powershellgallery.com/api/v2/package/Pester/3.4.0",
   "LicenseUrl":"http://www.apache.org/licenses/LICENSE-2.0.html",
   "Version":"3.4.0",
   "CacheVersion":"2.4"
}

[forki]

the issue is that nuget.exe can interfere here. It it put's that pckage in the cach and overwrites the one from the other source then there is no way to know that.

[agross]

I just tested,

nuget install Pester

and

nuget install Pester -source https://www.powershellgallery.com/api/v2 

do the right thing, regardless of the order they're called (even with a primed cache).

[agross]

That also means that the cache pollution you are talking about already exists and Paket should make sure the JSON matches the cached nupkg.

[forki]

do the right thing, regardless of the order they're called (even with a primed cache).

What does this mean? It verifies hashes?

[forki]

or what is happening in the cache folder?

[agross]

What does this mean?

Even if the nuget.org variant is in the cache, it downloads the PSG variant and puts it in the working directory and the other way around.

[forki]

would be nice to know how it decides that.

[Stift]

Does it make sense to have a cache per source? %localappdata%\Nuget\Cache\hash(source)

This would solve the problem and we don't have to mess around with the json files.

[forki]

yes it would solve that, but then we would not share with nuget.

[Stift]

nuget.org could be an exception!

[Stift]

I mean the most people have only nuget.org, the others have different sources and need to distinguish, esp. when two distinctive sources (purpose-wise) come in.

[forki]

mhm, that sounds doable. (There is still the case left tha nuget puts the wrong package in the root and we don't see that, but I think that's minor)

Still would be nice to know how they decide. The nuget.org hash can't be trusted

[Stift]

did not know that. Is there an example for that?

[forki]

we tried in early days of paket - that was total disaster

[Stift]

And it was there before... just an addition - we could put an source.txt in the %localappdata%\Nuget\Cache\hash(source) to see where this cache folder belongs.

[enricosada]

based on https://github.com/fsprojects/Paket/issues/1287 feedback:

Include hashes in lock file and check at restore

[xavierzwirtz]

I am looking at implementing this to enable Nix derivations(basically a package) to be generated from a paket.lock file. Would there be interest in that pr? Ideally it would be enabled by default, to enable maximum penetration into the ecosystem. I don't need verification that the file hash of downloaded files is correct, just writing the hash of the nupkg file when building the lock file.