Closed bartelink closed 8 years ago
http://fsprojects.github.io/Paket/nuget-dependencies.html#Path-sources this would probably work
@forki But I dont want to manage such a mirror.
I want the packages in the packages
folder. I want to check them into source control. But I don't want the DLLs and/or expanded content checked in.
And Paket has all the info/control it needs to work from the packages folder without any such mucking about and indirection on my behalf.
just do the following:
nugets
in your solution rootsource ./nugets
still add /packages to gitignore.
Done
And depending on the maturity of a project and/or the contributors' styles, one might just rely on NuGet caching for a bit and then 'flick the switch' by saying paket init-compact-mode
, which immediately makes n .nupkg
s ready to git commit -a -m "Now works on the plains and the planes"
@forki But
/nugets
.paket update
know to write through
the 'cache' that /nugets
represents?/nugets
?@forki that's exactly what I'm doing right now, but it would be awesome if Paket could automatically maintain the nugets
folder for me, i.e. automatically mirroring the used packages from a remote feed.
so what you want is a second cache level which contains only installed packages and is customizable?
@forki No. Paket needs Pakets.
It needs them to install
from.
When it update
s, it needs to coordinate the contents of the folder
There is zero value in having 2 copies of the nupkgs and/or the potential for them to lose sync.
The NuGet cache/download pipeline will, over time (as you've already accomodated) offer different resiliency options.
The download schemes from github / fssnip can do their own stuff too.
Paket has the overview of all dependencies and I'm not trying to introduce any new thing requiring new work. If you re-read the main post and/or I fix whatever confused you, you'll see... :D
The proposal is, in summary:
.gitignore
convention.proj
which detects a canary / marker which signifies "don't expect the DLLs"Some of my conventions presuppose some of the architecture in #154 in terms of unifying all the other things BUT right now, this scheme is extremely implementable for NuGet packages without any drastic messing about in the core impl of Paket.
OK, no more editing this comment!
UPDATE: cut/paste of side conversation
so what you want is a second cache level which contains only installed packages and is customizable?
I think second level cache describes it well But we're using it as the one and only source for pacakge restores/installs
It will have the effect of being a second level cache
But its a primary thing which will be doing double duty
caches keep copies and coordinate stuff
I don't see why package restore and your .gitignore convention won't work together.
@forki As in NuGet Package Restore ? Ah, might just work! .. HEY WAIT - there's no packages.config
so how would/does VS know to unzip the .nupkg
files first?
(Was typing) Of course, if someone can think of a way for (the same way new versions of NuGet work) for VS to trigger a paket install
before attempting to load, we'd be golden and wouldnt need to emit such a message.
OK, so I don't know my .gitignore
inside out - what's the simplest way of expressing "Include only .nupkg
files directly under packages/
. And exclude all other files under packages."
I assume that can be expressed as a single one liner rule. Does it make sense for Paket to look at .gitignore
or is that a step too far? Or is it a doc / FAQ issue?
@vasily-kirichenko @rneatherway any ideas ?
1) we should check if package folder is empty (except for nupkg) then we need to unzip (have to check. Maybe we already do this) 2) I don't know of a way to trigger MSBuild targets on VS load. But I don't think we really need this. 3) I prefer to have a separate .MD for this since I think a lot of people will like it
@forki 1: Concievably a .nupkg
could jump in from a branch switch so important to guarantee consistency
2: Re MSBuild targets main thing is to have a consistent can clear message that is searchable and/or points to doc and tells them to run paket install
In my experience its absolutely critical for people to never end up with an inconsistent project load as people are not always awake and/or understand WTH is going on in proj files.
But it def doesnt need to be high-tech. Just predictable - breaking, not bending if you will :)
3: Separate MD is cool with me - its def a concept that needs to be laid out with examples that I'm clearly not in a position to give!
And now, about that #154 elephant in the room :P
can anyone try to figure out the gitignore pattern? /cc @theimowski
Probably:
!.nupkg
Alexander Groß Tiny phone, tiny mail
On Thu, Sep 25, 2014 at 2:04 PM, Steffen Forkmann notifications@github.com wrote:
can anyone try to figure out the gitignore pattern? /cc @theimowski
Reply to this email directly or view it on GitHub: https://github.com/fsprojects/Paket/issues/166#issuecomment-56808969
Sorry for the brevity of previous comments, me and @bartelink were just throwing ideas around. Maybe we didn't even mean the same thing.
@forki I'm not sure whether it's a good idea to checking things from the packages
folder. I think you were close by saying a "second level cache" with the nugets
folder, but I think a better description may be a "curated, local mirror" of the packages from another feed.
Then you put something like this in your NuGet.config next to your solution (.sln)
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="local-feed" value="lib/nuget-feed" />
</packageSources>
<activePackageSource>
<add key="local-feed" value="lib/nuget-feed" />
</activePackageSource>
</configuration>
I think nuget restore should automatically pick it up, if not explicitly pass it via -ConfigFile
option to the nuget restore
command. So on restore, you are retrieving all packages from that curated, local mirror. It's not only faster than using the official nuget feed, but also a lot more reliable (and gives you more reproducible builds as well when indirect dependencies are involved, which is the whole idea of Paket if I got that right).
Using the curated, local mirror approach you can keep using package restore like normal. That's how I do it currently, but I have to manually copy the nupkgs to my feed mirror directory. That's what I want paket to automate.
Now, if paket does already unpack/install nupkgs, committing them to the packages directroy may be enough. The .gitignore would then need to exclude every 2nd level subdirectory and below in the packages folder. With mercurial and glob syntax (not a git fan here, so may need translation):
packages/*/*/**
That leaves the nupkg files and the odd readme/license. I did actually try that approach first, committing packages//.nupkg, but that didn't play well using nuget restore, which would clame it had done its job while not unpacking the nupks.
So, as I see it, there are two choices for paket: 1) help maintaining the curated, local mirror feed 2) unpack nupkgs before build (which would probably require an msbuild hook somehow....)
Personally, I'm leaning towards the former.
I only want to exclude them from within packages
2014-09-25 14:19 GMT+02:00 Alexander Groß notifications@github.com:
Probably:
!.nupkg
Alex
Alexander Groß Tiny phone, tiny mail
On Thu, Sep 25, 2014 at 2:04 PM, Steffen Forkmann notifications@github.com wrote:
can anyone try to figure out the gitignore pattern? /cc @theimowski
Reply to this email directly or view it on GitHub: https://github.com/fsprojects/Paket/issues/166#issuecomment-56808969
— Reply to this email directly or view it on GitHub https://github.com/fsprojects/Paket/issues/166#issuecomment-56810306.
Put the gitignore in the packages directory. Or did I misunderstand you?
Alexander Groß Tiny phone, tiny mail
On Thu, Sep 25, 2014 at 2:34 PM, Steffen Forkmann notifications@github.com wrote:
I only want to exclude them from within packages 2014-09-25 14:19 GMT+02:00 Alexander Groß notifications@github.com:
Probably:
!.nupkg
Alex
Alexander Groß Tiny phone, tiny mail
On Thu, Sep 25, 2014 at 2:04 PM, Steffen Forkmann notifications@github.com wrote:
can anyone try to figure out the gitignore pattern? /cc @theimowski
Reply to this email directly or view it on GitHub: https://github.com/fsprojects/Paket/issues/166#issuecomment-56808969
— Reply to this email directly or view it on GitHub https://github.com/fsprojects/Paket/issues/166#issuecomment-56810306.
Reply to this email directly or view it on GitHub: https://github.com/fsprojects/Paket/issues/166#issuecomment-56811847
but I want a global gitignore :-)
Not a good idea, IMHO. Global breaks when you move things around. Violates DRY ;-)
Does this work?
packages/ !packages/*.nupkg
Alexander Groß Tiny phone, tiny mail
On Thu, Sep 25, 2014 at 2:37 PM, Steffen Forkmann notifications@github.com wrote:
but I want a global gitignore :-)
Reply to this email directly or view it on GitHub: https://github.com/fsprojects/Paket/issues/166#issuecomment-56812129
@JohannesRudolph Lots of things to clarify. #125 makes a proposal to keep sources in a central place in a way that alludes to what you describe.
But the most fundamental thing here is that paket convert-from-nuget
- the main way Paket makes sense to use:
paket convert-from-nuget
packages.config
files and replaces them with paket.references
which list just file names /packet.dependencies
paket install
looks at the lock file and
packages/*.nupkg
s it can't see that the packages.references
need according to the versions locked in paket.lock
(but if the files are there it just uses them as paket update
is entrusted with responsibility of getting all files consistent.csproj/fsproj/vbproj
files to ensure all references are rightpaket update
conceptually
paket.lock
filepackages
directory - it knows everything requiredThe proposal is that
.nupkg
files are already there but not the DLLspaket install
. The ONLY reason for this is for "clone in VS people"There is no reliance on NuGet Package Restore and/or what versions 1-3 do in 9 releases of VS since VS2010SP1.
The other thing is that Paket needs to own an autoritative set of creds and feed locations across all feeds inc fssnip, github, local files etc.
BUT this is only used in 2b. Nothing else cares about the feeds. Paket Restore just works of a set of Pakets.
In step 1 of #154, "set of Pakets" means
.nupkg files
in the (unversioned) subdirs under /packages/
In step 2 of #154, "set of Pakets" means
.nupkg files
in the (unversioned) subdirs under /packages/
.nupkg
files too*@JohannesRudolph Based on the prev comments, I'd respond to your post as:
Sorry for the brevity of previous comments, me and @bartelink were just throwing ideas around. Maybe we didn't even mean the same thing.
V likely :P
Then you put something like this in your NuGet.config next to your solution (.sln) OW, OLDE STYLE ANGLY BWAKETS, WE HAVE PARSERS AND DSLs!
No mirror defs
I think nuget restore should automatically pick it up, if not explicitly pass it via -ConfigFile option to the nuget restore command.
Nobody will be using NuGet restore.
So on restore, you are retrieving all packages from that curated, local mirror.
2b put them in packages
. If you committed to source control they are there already. If not, 2b in a paket install
will get them again.
It's not only faster than using the official nuget feed, but also a lot more reliable (and gives you more reproducible builds as well when indirect dependencies are involved, which is the whole idea of Paket if I got that right).
Have you tried Paket? There is nothing in NuGet's restore path that compares to the paket install
perf.
The builds are reproducable as you say coz 3 computed the graph and stashed it in the paket.lock
file
Using the curated, local mirror approach you can keep using package restore like normal. That's how I do it currently, but I have to manually copy the nupkgs to my feed mirror directory. That's what I want paket to automate.
Why? Does it handle multiple sources? If I am using an authenticated feed do I need to stash junk on my build machine? Does that accomodate non-NuGet sources?
Now, if paket does already unpack/install nupkgs, committing them to the packages directroy may be enough. The .gitignore would then need to exclude every 2nd level subdirectory and below in the packages folder. With mercurial and glob syntax (not a git fan here, so may need translation):
You didnt put ticks around angle brackets, I guess?
That leaves the nupkg files and the odd readme/license. I did actually try that approach first, committing packages//.nupkg, but that didn't play well using nuget restore, which would clame it had done its job while not unpacking the nupks.
Any Package restore stuff is stripped by paket convert-from-nuget
. The block 2c puts in should hopefully neutralize any madness it wants to do. I agree the right thing needs to happen and we don't want VS trying to help and the causing a mess/
So, as I see it, there are two choices for paket: 1) help maintaining the curated, local mirror feed
No mirrors. No caches. No cache invalidation. No curation. Persistence of outcome of a deterministic download process. With a priviso for the stuff to already be there due to commiting to source control.
2) unpack nupkgs before build (which would probably require an msbuild hook somehow....)
@forki Could write a book NP. (And I've typed a book worth in contravention of @shanselman 's calls).
MORE IMPORTANTLY: Paket does all the stuff to unzip, edit the projs etc. In a flash. @forki has implemented the entire process, hard as that may be to imagine (I have to say I had the same reaction in terms of not considering ny of this stuff as being remotely feasible, but the counterexample is right here in the repo). (To be clear: there are plently other contributors and it might have taken an extra while without them, but the core of thie impl of this aspect is from @forki)
Personally, I'm leaning towards the former.
Still ? Need to do a skype call if so :P
@agross I was fearing that might be the answer :( Maybe a minion needs to get Just One More Little Feature into git if you are right :P
ok I tried. I don't get gitignore to work. Even if I put it into subdir.
If anyone want's to try it...
@forki try this (see my above answer but I had the markdown wrong so it didn't show up). Works with mercurial.
packages/*/*/**
@forki When Paket unzips, it lands the .nuspec
s and misc files such as [Content_Types].xml
beside the .nupkg
It would seem to me that @JohannesRudolph 's scheme can work if the install aspect consumes the .nuspec
from the .nupkg
on the fly and not send it out to disk (it's redundant anyway, or is it important for performance / cross checking of versions in early phases of the process) ?
see also (possible) discussion (later) today https://jabbr.net/#/rooms/fsharp
Prompted by an IM from @JohannesRudolph:
I'd like to see alongside the Paket restore mode, a mode in which Paket in a consistent way:
.nupkg
files (downloaded source files would be serialized as.nupkg
s)[https://github.com/fsprojects/Paket/issues/114#issuecomment-56251073]packages/*.nupkg
is saved (the presence of the DLLs etc. is guaranteed idempotently bypaket install
) (but not subdirs - so it can tell if the files are not loaded, see below)This could detect that the packages are not expanded (due to
.nupkg
files only being in/packages
, and the canary one in/packages/.paket/expanded.nupkg
not being present).This would offer a middle way, addressing
I think an important thing is that in this mode, loading in VS is bad news - VS will silently half-load references and various other things and/or display other inconsistent behavior where DLLs do not exist.
Thus if each project had a target and/or failing import /
InitialTargets
or similar which could do a:<Error Text="This project uses Paket compacted mode (See http://). Please use
paket installto prepare the repository for use from Visual Studion" Condition=" EXISTS( '$(SolutionRoot)/packages/.paket/expanded.nupkg' ) />
(Maybe just an Import with the message above as a filename?)