fsprojects / Paket

A dependency manager for .NET with support for NuGet packages and Git repositories.
https://fsprojects.github.io/Paket/
MIT License
2.02k stars 524 forks source link

ability to specify which sources to pull the dependencies from #1320

Closed ctaggart closed 6 years ago

ctaggart commented 8 years ago

Paket looks for the nuget dependencies in all of the sources for a group. It would be great if we could limit the sources used to lookup the dependencies. This change would increase performance and reduce the unnecessary load that is put on NuGet feeds.

Using this as an example, I named the sources and added from lists. https://github.com/ctaggart/SourceLink/blob/master/paket.dependencies

source nuget https://www.nuget.org/api/v2
source appveyor https://ci.appveyor.com/nuget/ctaggart-sourcelink

from nuget appveyor
  nuget SourceLink.Fake 1.1.0-b279

from nuget
  nuget FAKE
  nuget FSharp.Formatting
  nuget NuGet.CommandLine
  nuget RazorEngine
  nuget FSharp.Core
  nuget LibGit2Sharp
  nuget Mono.Cecil
  nuget UnionArgParser
  nuget System.Threading.Tasks.Dataflow
  nuget Argu

Performance related tickets this would help are #1317 & #1249.

forki commented 8 years ago

Our first implementation did it like this, but unfortunately for transitive dependencies we need to change the source. On Dec 19, 2015 11:45 PM, "Cameron Taggart" notifications@github.com wrote:

Paket looks for the nuget dependencies in all of the sources for a group. It would be great if we could limit the sources used to lookup the dependencies. This change would increase performance and reduce the unnecessary load that is put on NuGet feeds.

Using this as an example, I named the sources and added from lists. https://github.com/ctaggart/SourceLink/blob/master/paket.dependencies

source nuget https://www.nuget.org/api/v2 source appveyor https://ci.appveyor.com/nuget/ctaggart-sourcelink

from nuget appveyor nuget SourceLink.Fake 1.1.0-b279

from nuget nuget FAKE nuget FSharp.Formatting nuget NuGet.CommandLine nuget RazorEngine nuget FSharp.Core nuget LibGit2Sharp nuget Mono.Cecil nuget UnionArgParser nuget System.Threading.Tasks.Dataflow nuget Argu

Performance related tickets this would help are #1317 https://github.com/fsprojects/Paket/issues/1317 & #1249 https://github.com/fsprojects/Paket/issues/1249.

— Reply to this email directly or view it on GitHub https://github.com/fsprojects/Paket/issues/1320.

ctaggart commented 8 years ago

I'm not clear what wouldn't work here with transitive dependencies. May be we can work through a better example.

forki commented 8 years ago

Sourcelink.fake might have a dependency on something that is on the other feed. On Dec 20, 2015 12:12 AM, "Cameron Taggart" notifications@github.com wrote:

I'm not clear what wouldn't work here with transitive dependencies. May be we can work through a better example.

— Reply to this email directly or view it on GitHub https://github.com/fsprojects/Paket/issues/1320#issuecomment-166032908.

ctaggart commented 8 years ago

The logic I was imagining, but didn't write down allows for that. It doesn't, but let's pretend SourceLink.Fake had a dependency on canopy. canopy then depends on FSharp.Core, Selenium.WebDriver, Selenium.Support, and SizSelCsZZZ. In the above example, I put down:

from nuget appveyor
  nuget SourceLink.Fake 1.1.0-b279

meaning that it should look for SourceLink.Fake in both the nuget feed and appveyor feed for it and its dependencies. After finding Canopy in that list of feeds, it would look for its transitive dependencies in that list of feeds unless it is a named dependency. In this example, FSharp.Core is a named dependency that should be looked up in only the nuget feed.

In summary, a named dependency has a list of sources to look its dependencies. Unnamed transitive dependencies inherit that list. Does that work? What are the edge cases that I'm missing?

forki commented 8 years ago

yes that would work and is basically what we had earlier. but in most common scenarios basically everything went under the complete list

ctaggart commented 8 years ago

Private NuGet feeds are really common for any business. Besides reducing performance and causing unnecessary load, I don't like that it queries all other feeds for our private packages. I'll use Gloun as an example (but it is now open source). image

I think this would be a really great feature. I image the update that now takes 51 seconds in #1317 would drop to 10 seconds. The tricky scenario would be when the same transitive dependency shows up under different from sources. Logically, you should just use all of those sources.

aggieben commented 7 years ago

I ran into a situation recently (yesterday) in which having this feature would have been helpful. I wanted to reference a nupkg built from a git source, but one that has the same name (and version) as one published on nuget.org. So, like this:

source <nuget v3 feed>
nuget PackageName

git <url to repo> <commit id> build: <command>, Source: /relative/path/to/nupkgs/, OS: windows
nuget PackageInGitRepoButAlsoOnNugetFeed

It would have been really nice to do something like

source nuget <nuget v3 feed>
git thesources <url to repo> <commit id> build: <command>, Source: /relative/path/to/nupkgs/, OS: windows

nuget PackageName
from thesources
  nuget PackageInGitRepoButAlsoOnNugetFeed

# or alternatively
nuget PackageInGitRepoButAlsoOnNugetFeed  from: thesources
matthid commented 7 years ago

In a enterprise situation you should probably mirror your packages into your own nuget server and/or make it relay/proxy/cache on the serverside.

@aggieben Same versions with different content should basically never exist, they lead to huge problems everywhere at the moment.

ctaggart commented 6 years ago

Paket has supported groups with their own sources now, which is what I wanted originally.