How does Paket stack up against NuGet 3's project.json?

[SamB]

I notice that the Paket website/documentaton doesn't make any mention of project.jsonProject.json Intro, which for NuGet's purposes basically replaces package.config (in participating projects), with a few differences:

1. First and foremost, project.json does not bake in transitive dependencies; instead, these are resolved during package restore and written into a lockfile. (This should sound familiar.)
2. With project.json, NuGet does not modify project files during package "install" or restore (Except if project.json is the project file, like for ASP.NET 5, though reportedly NuGet is not actually used for ASP.NET 5?) Instead, the build system is supposed to pick up references from the lockfile, project.lock.json.
3. Install/Uninstall scripts are unsupported and ignored by design. (Also familiar.)
4. Content transforms are unsupported and ignored by design. (Presumably, whether or not this sounds familiar, Paket behaves the same way.)
5. Content files aren't supported either, but in this case that's not by design, it's because they haven't decided on a design for how this should work with project.json yet.

And that's just the things that NuGet 3 doesn't do anymore with project.json; there are also some new features, too.

It would be nice if you'd talk about the extent to which the advantages that you mention Paket has over packages.config also apply to project.json, whether any new advantages have cropped up, and (god forbid!) any advantages project.json might have at the moment.

Docs

The NuGet documentation includes these pages about project.json:

• Project.json Intro, or NuGet 3.0, .NET and project.json
• Project.json Format, or Project.json Usage
• Project.json Dependency, or Dependency Resolution in NuGet v3 / project.json

[forki]

Since I'm still trying to understand the workflow in DNX and coreclr the following might not be correct.

This the structure I think will be used in coreclr/DNX:

• Solution.sln
• Project1/
  • project.json
  • project.lock.json
• Project2/
  • project.json
  • project.lock.json

Every project.json contains a dependencies block like:

"dependencies": {
    "EntityFramework.Commands": "7.0.0-beta8",
    "EntityFramework.SqlServer": "7.0.0-beta8",
    "Microsoft.AspNet.Authentication.Cookies": "1.0.0-beta8",
    "Microsoft.AspNet.Authentication.Facebook": "1.0.0-beta8",
    "Microsoft.AspNet.Authentication.Google": "1.0.0-beta8",
    "Microsoft.AspNet.Authentication.MicrosoftAccount": "1.0.0-beta8",
    "Microsoft.AspNet.Authentication.Twitter": "1.0.0-beta8",
    "Microsoft.AspNet.Diagnostics": "1.0.0-beta8",
    "Microsoft.AspNet.Diagnostics.Entity": "7.0.0-beta8",
    "Microsoft.AspNet.Identity.EntityFramework": "3.0.0-beta8",
    "Microsoft.AspNet.IISPlatformHandler": "1.0.0-beta8",
    "Microsoft.AspNet.Mvc": "6.0.0-beta8",
    "Microsoft.AspNet.Mvc.TagHelpers": "6.0.0-beta8",
    "Microsoft.AspNet.Server.Kestrel": "1.0.0-beta8",
    "Microsoft.AspNet.StaticFiles": "1.0.0-beta8",
    "Microsoft.AspNet.Tooling.Razor": "1.0.0-beta8",
    "Microsoft.Framework.Configuration.Abstractions": "1.0.0-beta8",
    "Microsoft.Framework.Configuration.Json": "1.0.0-beta8",
    "Microsoft.Framework.Configuration.UserSecrets": "1.0.0-beta8",
    "Microsoft.Framework.Logging": "1.0.0-beta8",
    "Microsoft.Framework.Logging.Console": "1.0.0-beta8",
    "Microsoft.Framework.Logging.Debug": "1.0.0-beta8",
    "Microsoft.VisualStudio.Web.BrowserLink.Loader": "14.0.0-beta8"
}

and the project.lock.json contains huge amount of lines with resolved dependencies.

Now to your questions/observations:

1. "project.json does not bake in transitive dependencies; instead, these are resolved during package restore and written into a lockfile."

   Yes, the wording sounds familiar to what we do with paket.dependencies/paket.lock, but it's completely different. The generated project.lock.json of a new ASP.NET beta 8 project (basically empty) is 19705 lines and therefor the somewhat official recommendation is to gitignore it. This means project.lock.json is not the same kind of lock file like paket.lock. paket.lock is usally commited and used to pin all versions (incl. transitive dependencies). Since project.lock.json is rebuild on restore you don't pin transitive dependencies and therefor there is a big chance that you get a different resolution on different machines or if just wait a bit until new versions are released. This makes it impossible to have reproducible builds. Just image you test something on your machine - everything works. CI is restoring different versions and product breaks in production. Happy times.

   Of course you could just commit project.lock.json, but then prepare for worst merge conflict hell you can image. (One small addition. I tried to do this in an experiment, but it always restored from project.json. I didn't find a way to avoid that, yet.)

2. "NuGet does not modify project files during package "install" "

   For me project.json looks like is basically part of the project system. As shown above it contains the top-level dependencies incl. version numbers. If you "install" then this file gets touched. If you "update" a package, this file gets touched. If two devs update at same time, you get merge conflicts in ALL of these project.json files. Since the versions are not globally maintained you can introduce unwanted version conflicts between two projects like packages.config model allowed you. AFAIK there are ways to use floating dependencies here, which could help to . But then you don't even lock the top-level dependencies and you will have even less reproducible builds.

   a) "though reportedly NuGet is not actually used for ASP.NET 5"

   AFAIK they use "dnu", but it's very similar tool with same model.

3. "Install/Uninstall scripts are unsupported and ignored by design"

   That's a result of the fact that we want to be on other platforms where Powershell doesn't work. IMHO it was a bad idea to introduce this in the first place and not make installation completely declarative. But NuGet team was telling us this for a long time that this feature will be removed at some point.

4. "Content transforms are unsupported and ignored by design"

   Here I'm not sure why this is. I assume dnu is just not ready implementing it. In Paket we also have an open feature request.

5. Content files aren't supported either

   Paket supports content files like old NuGet (but with some bugs maybe - who knows).

My conclusion at this point is that the new NuGet model for coreclr and DNX still has most of the flaws that we have see in older NuGet versions. But with the dependencies block and the restore of project.lock.json we see new much more serious issues. If I really understand that model and you don't pin all package versions then this is a recipe for disaster.

TBF there exists nice tooling around project.json which makes it look like a huge improvement - especially compared to the NuGet VisualStudio addin. But that doesn't really help if you deploy basically random unknown versions of transitive dependencies.

Regarding Paket: I think Paket's paket.dependency/paket.lock model is needed more than ever, but it's hard to make it work (see https://github.com/fsprojects/Paket/issues/736 and https://github.com/Microsoft/visualfsharp/pull/722).

PS: I didn't even look into how F# script files can work in this model.

[khellang]

AFAIK they use "dnu", but it's very similar tool with same model.

dnu will die and be replaced by nuget, or even the new dotnet unified CLI experience at some point. This was just an interim utility because NuGet didn't understand project.json.

Here I'm not sure why this is. I assume dnu is just not ready implementing it. In Paket we also have an open feature request.

This is because the current content transform design was pretty bad. It only worked in C# project types and had no way to pivot on language. The latter applies to content files in general.

There are plans on bringing this back, with a better way of controlling where the content files ends up in your project structure, preferably by not copying the files at all, but rather just "linking in" a read only version. It'll also let you pivot on language and target platform, so you can include different files for js/fs/cs/vb etc. and specify the content files' build action.

Most of the goals for this is outlined at https://github.com/NuGet/Home/wiki/Bringing-back-content-support,-September-24th,-2015

[forki]

@khellang thanks for this clarification.

@khellang just showed me https://github.com/NuGet/Home/wiki/Lock-file-design-meeting-notes---August-18,-2015

The "snapshot file" would probably solve some of the issues. It would still allow you to accidently use different versions in different projects but it would at least allow you to pin versions and have reproducible builds.

If the dependencies block in project.json would be made optional and the dnx/nuget tooling that generates project.json.lock would just look at the "snapshot" file, then it would also opens a chance for Paket to work with coreclr and DNX.

Paket could generate these snapshot files and paket users could gitignore the snapshot files and the resulting project.lock.json files.

[khellang]

I think I saw somewhere that someone proposed a "dependencies": "external" thing, where you could specify a 3rd party command to run. That might be a solution. That external tool's task would be to simply create a "snapshot file" and return a success exit code.

[forki]

Yes that was in #736, but the suggestion was to create project.json.lock which seems like overkill. Creating the snapshot file would be much nicer.

[davidfowl]

So after thinking about this a bit, you really don't need to do anything with project.json or project.lock.json to be core clr friendly. These things can all work with msbuild the way that .NET Framework does today. The project.json model is a new project system with an integrated experience that we built. paket still works great with msbuild based projects so my question is really, does it need to work with project.json based projects as well?

[forki]

The point is MS is pushing this new model and even projects like the F# compiler are changing (very slowly in that direction). We want to allow our users to benefit from all the shiny new things, but still allow them to do dependency management like they like it.

With the "snapshot" files (which I believe are needed anyways - see above) and optional dependencies in project.json this could work beautifully.

[isaacabraham]

@davidfowl I'm confused. Doesn't @forki at the top of this issue show where the differences between nuget 3 / project.json and paket are? I'm sure that a lot of great work has been done in building an integrated experience, but the coupling between nuget and project.json is extremely tight to the extent that it seems like everyone is being forced down this route in the new world of project.json. This is a real shame, because as been pointed out, there are some fundamental differences in opinions of how dependency management should work.

I'm not going to rehash them all again because not only are they stated above but also discussed at length in #736.

[GerjanOnline]

@davidfowl Please just make the snapshot file happen and everyone is happy again.

I only see advantages here:

• "Real" reproducable/repeatable builds (I totally agree with @Mpdreamz about this in https://github.com/aspnet/dnx/issues/2332, it's a crucial thing)
• Support for other package managers like Paket (which implicitly means support for other communities like F# imo)
• Looser coupling between the project system and package manager/resolution
• Everyone can use the new shiny project.json (without the deps)

[davidfowl]

@forki You didn't actually answer my question.

"Real" reproducable/repeatable builds

Totally doable already with the tech as it exists today. I think there's tons of FUD around this personally because people want it to work a specific way.

Support for other package managers like Paket (which implicit means support for other communities like F# imo)

There's only Paket and NuGet. I'm trying to understand why you paket needs to fit into project.json land (without a csproj), ms isn't pushing anything. There's a new project system that is integrated with the package manager (just like every most other platforms do, like rust and cargo for e.g.). There's also the existing msbuild project system that is super decoupled and powerful and works with paket already.

Looser coupling between the project system and package manager/resolution

Except the point of this project system is to couple package building and producing. Decoupling it is strictly a non-goal. This isn't about trying to block paket from working and there is no evil intent here. The original goal of project.json was to unify people producing and consuming packages so that they experience was one and the same. That's what we achieved with project.json and it's why projects like MVC can build packages without the need for a csproj and nuspec (https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNet.Mvc.Core/project.json).

Everyone can use the new shiny project.json (without the deps)

That defeats the purpose. If you're not going to use the integrated experience there's no point going to this model.

@davidfowl I'm confused. Doesn't @forki at the top of this issue show where the differences between nuget 3 / project.json and paket are? I'm sure that a lot of great work has been done in building an integrated experience, but the coupling between nuget and project.json is extremely tight to the extent that it seems like everyone is being forced down this route in the new world of project.json. This is a real shame, because as been pointed out, there are some fundamental differences in opinions of how dependency management should work.

You are being forced down this route the same way NuGet *forces you to use packages.config. Our tooling understands the project format because we made it. Like I said, you don't have to use it to make CoreCLR work. That looks as if it is the end goal and that is doable while maintaining all of paket's features. If you could articulate what it is you want specifically from the project.json that would help, because 90% of what we tried to simplify was dependency management.

[isaacabraham]

@davidfowl ok - perhaps I have massively misunderstood project.json. If I have, that's my bad. My understanding is that project.json is a replacement for e.g. .csproj, but also brings into the game an understanding of packages as well.

If so - does that not mean that project.json is about more than just dependency management? And therefore if Paket wanted to go down the road of not using project.json, it would have to redo all the other things that project.json brings to the party.

I read that ASP .NET 5 (and UWP?) will use project.json by default. If that is the case, then if you want to use Paket, you need to again go outside the standard route and completely replace the whole system, including any tooling that will be built to work with project.json.

If I'm wrong on any of this, please tell me.

[FransBouma]

@davidfowl

Totally doable already with the tech as it exists today. I think there's tons of FUD around this personally because people want it to work a specific way.

Could you point me to a page/article/post which proves builds are always repeatable with the new project system of yours? That would be great. MS saying it's FUD is nice, but it's nothing more than hearsay ;)

That defeats the purpose. If you're not going to use the integrated experience there's no point going to this model.

So does this effectively mean: management of the project pipeline is done by your tooling and your tooling alone, doing that through 3rd party tooling is not really recommended/supported?

[nosami]

Like I said, you don't have to use it to make CoreCLR work

project.json is needed to make DNX work though.

[jeroldhaas]

What are the cons of implementing the snapshot idea, if any? Seems a great deal of push-back against a request for an open design.

[davidfowl]

I read that ASP .NET 5 (and UWP?) will use project.json by default. If that is the case, then if you want to use Paket, you need to again go outside the standard route and completely replace the whole system, including any tooling that will be built to work with project.json.

In UWP applications, there's still a csproj file that handles everything else (compilation, build pipeline etc). Paket still works there. The project.json there is only used for dependencies, it's equivalent to packages.config but supporting transitive dependencies. MSBuild is also now NuGet aware. For these types of projects, there's a new msbuild task that understands how to parse the project.lock.json file format and turn it into msbuild items at build time. This is equivalent of what Paket does today manually using the imports file (or by editing the project file).

DNX projects (ASP.NET 5 projects) try to go a step further and make NuGet the only experience. It is coupled on purpose. Projects and packages are one in the same, consuming a package from a "DNX project" is a step away from producing a package with that package as a dependency. In this model, the project.json acts as the nuspec, the csproj file and the packages.config. This is the design of the new system. In a way, we started from a nuspec and reversed engineered a project format. This is in the spirit of how other platforms and systems work where the package manager is at the heart of the experience.

As part of that reverse engineering, we needed to be able to express compiler arguments since there was no msbuild to do the compilation (full list here):

https://docs.asp.net/en/latest/dnx/projects.html?highlight=working%20dnx

[isaacabraham]

Ok. Thanks for providing that info. I think that there are lots more questions that are coming regarding this e.g. What happens for other project types adopting this in future etc.

Just so I understand - UWP projects are potentially doable (effectively packages.config is replaced with project.json). If MSBuild is now NuGet aware, does that suggest that there will be tooling built into e.g. VS in the future will be tied into that as well?

ASP .Net 5 projects sound like they won't ever be Paket enabled as they are specifically designed to work with NuGet only.

Is that a correct interpretation? If so, then Paket can try to move forward as best as it can to work with as much as possible, where possible.

[FransBouma]

ASP .Net 5 projects sound like they won't ever be Paket enabled as they are specifically designed to work with NuGet only.

... isn't that the core problem? I mean, why is there an inner focused 'our stuff or nothing at all' mentality? Isn't it better to have a dialog about what's needed by 3rd party tooling to work with what's designed for ASPNET so people have choices?

[SamB]

Hmm, I only meant to refer to "project.json the package.config replacement" (as I've seen used in msbuild and rosyln's source trees, and as @davidfowl says UWP projects use, with .csproj or .vbproj files that mainly differ from the typical in that they do contain explicit references to assemblies from the packages that they depend on), not "project.json the project format".

(Incidentally, would anyone happen to know where the "NuGet.MsBuild.Integration" package comes from?)

[davidfowl]

ASP .Net 5 projects sound like they won't ever be Paket enabled as they are specifically designed to work with NuGet only.

I don't want to say never because things can always change. If we design some package manager extensibility, it has the fit and gel with the model. We're not going to hack something into the system as a quick way to make paket work. IMO paket was designed to make the pain of NuGet as it existed in a previous form go away. It does a few things in particular very differently to NuGet and there other general things that would need to be fleshed out experience wise to make the tooling work well.

The sky is not falling, and we care about F#, that's why this https://github.com/fsprojects/fsharp-dnx/ exists.

[Mpdreamz]

"Real" reproducable/repeatable builds

Totally doable already with the tech as it exists today. I think there's tons of FUD around this personally because people want it to work a specific way.

Nobody is in the business of spreading FUD here. I'll refrain from reiterating for the umpteenth time why "doable" does not equate "real".

I would love to see an updated nuget design meeting with some of the feedback microsoft provided after august e.g https://github.com/aspnet/dnx/issues/2332#issuecomment-143874461 which has sadly been bumped from rc-1 to the backlog of 1.0. /cc @csharpfritz

You mention cargo as an example, I hope you reread its locking section http://doc.crates.io/guide.html#cargotoml-vs-cargolock and at the very very least get rid of the pseudo lock ("lock" : true) in project.json and use the lock solely for resolved dependencies. Making that lock file generate by default, like cargo, would be awesome.

DNX projects (ASP.NET 5 projects) try to go a step further and make NuGet the only experience.DNX projects (ASP.NET 5 projects) try to go a step further and make NuGet the only experience.

The fact you are pushing back so hard on opening up the package manager portion of DNX's project.json reeks of hubris to be honest.

If you're not going to use the integrated experience there's no point going to this model.

This is the exact sentiment I had hoped we could collectively move away from in the .NET dev'o'sphere.

When DNX first started I was over the moon but its become painfully obvious that promise of a new simplified landscape 1+ year ago has turned into the exact opposite. Doing any DNX project will result in xproj/sln/project.json/global.json files littered in your code base. None of the nicely integrated project systems you've mentioned have this.

You equate DNX projects with (ASPNET5 projects) but is that really what it comes down to now? a glorified node like run time for web apps? This saddens me deeply, there is some amazing tech in DNX.

I'll take cargo as an example again, something is a binary or a lib thats it. This is the same promise I saw with DNX, no more project guids!, DNX even goes out of its way to make everything a lib which I personally do not like but alas. It's clear however it isn't simplifying squat since it has to live in the bigger VS world where we have UWP apps and others where old fashioned csproj files have to be used because really its only serving one use case.

None of the other project systems you mentioned have this IDE integration requirement and they are all better for it. Nor do they have alternating project formats based on the type of app being build.

You are being forced down this route the same way NuGet *forces you to use packages.config. Our tooling understands the project format because we made it. Like I said, you don't have to use it to make CoreCLR work. That looks as if it is the end goal and that is doable while maintaining all of paket's features.

This is all true, you don't need DNX to do CoreCLR. you don't need DNX to do MVC6. Does this mean there is no value for Paket to try and work with DNX? No, for all the bitching I do about the dependency management it really does have a nice project/compilation system, DTH rocks and it for sure will be the defacto platform for 95% of all web apps being written once it GA's. As stated earlier I sincerely hope Microsoft goes all in on DNX for more then aspnet in the future too. If we have to standardize on a project/complication system in the future i personally prefer DNX's project.json over csproj files.

[davidfowl]

Nobody is in the business of spreading FUD here. I'll refrain from reiterating for the umpteenth time why "doable" does not equate "real".

If we're getting down the facts here are the reason it's FUD:

• NuGet/DNU chooses the lowest version. Unless you delete packages from your feed, transitive dependencies DO NOT CHANGE.
• Paket prefers the higher versions so it requires an intermediate file in order to avoid floating.

The reason we don't need this file for repeatable builds is because dependencies don't float. You may disagree with that approach but it works.

I would love to see an updated nuget design meeting with some of the feedback microsoft provided after august e.g aspnet/dnx#2332 (comment) which has sadly been bumped from rc-1 to the backlog of 1.0. /cc @csharpfritz

Yes. Sadly we have like 1000 other bugs and this one got punted.

The fact you are pushing back so hard on opening up the package manager portion of DNX's project.json reeks of hubris to be honest.

It's like me saying that Paket should prefer lower versions. It's a design choice and principle Paket chose and who am I to question it? I can say it's a bad idea but there are reasons behind it. The same way I try to explain why we couple producing and consuming packages but nobody seems to care. They chalk it up to bad layering when in fact the net effect is pretty amazing (we've been living it for the last year).

When DNX first started I was over the moon but its become painfully obvious that promise of a new simplified landscape 1+ year ago has turned into the exact opposite. Doing any DNX project will result in xproj/sln/project.json/global.json files littered in your code base. None of the nicely integrated project systems you've mentioned have this.

Not sure I agree. Java has a bunch of IDE specific files when you use eclipse with different types of projects. That's no different from xproj. Also xproj exists for good reason, because we still have to interoperate with msbuild based projects and it gives us an easy way to do that. Global.json also exists for a reason and I don't see it as litter it's actually very intentional. These a files are very nicely integrated into both VS and omnisharp and the end to end experience. It's why you can swap out a package for source so easily, it was carefully designed to allow that.

One thing other systems don't have is the concept of "solutions" (aka multi project projects). Go has workspaces and that's the most similar thing I can think of. The other files are unique to the .NET landscape (VS solutions with > 100 projects, potentially unrelated).

This is all true, you don't need DNX to do CoreCLR. you don't need DNX to do MVC6. Does this mean there is no value for Paket to try and work with DNX? No, for all the bitching I do about the dependency management it really does have a nice project/compilation system, DTH rocks and it for sure will be the defacto platform for 95% of all web apps being written once it GA's. As stated earlier I sincerely hope Microsoft goes all in on DNX for more then aspnet in the future too. If we have to standardize on a project/complication system in the future i personally prefer DNX's project.json over csproj files.

I know I love it for that too (that's why we made it). Like I said before though, shoe horning Paket in it's current form doesn't seem like a good solution for me.

Can we change Paket to work differently for project.json based projects? Maybe the existing Paket is more for msbuild based projects and fits better but a newly designed version that was project.json friendly makes more sense?

[Mpdreamz]

If we're getting down the facts here are the reason it's FUD:

• NuGet/DNU chooses the lowest version. Unless you delete packages from your feed, transitive dependencies DO NOT CHANGE.
• Paket prefers the higher versions so it requires an intermediate file in order to avoid floating.

However nuget also allows floating prerelease package reference, so its already broken there. I am scared in time it will also support floating major,minor,patch versions. The GitHub issues asking for it on nuget already exist and who can blame them, having to lookup the latest version is tedious.

Choosing lowest also means that nuget favors ignoring patch releases that OSS authors might have issued. There's is nothing inherently safer choosing lowest vs highest, a lock file simply eliminates all doubts no matter what resolution strategy is chosen.

I think at this point we need to settle on the fact that we will never agree on this and that is OK, these two camps exist and it'd be nice if both can keep existing :+1: I'll drop the discussion on IDE files etcetera in the interest of staying on topic :tada:

Can we change Paket to work differently for project.json based projects? Maybe the existing Paket is more for msbuild based projects and fits better but a newly designed version that was project.json friendly makes more sense?

This intrigues me :) What specifc set of changes do you see required for paket to play nicely and not shoehorn its way around DNX? Up until now the discussion has been around DNX providing hooks for paket to do its thing but this flips it around.

[davidfowl]

However nuget also allows floating prerelease package reference, so its already broken there. I am scared in time it will also support floating major,minor,patch versions. The GitHub issues asking for it on nuget already exist and who can blame them, having to lookup the latest version is tedious.

I'd say lean on floating versions for your own dependencies (things that you control) and for 3rd party deps, just use the command that updates things.

Choosing lowest also means that nuget favors ignoring patch releases that OSS authors might have issued. There's is nothing inherently safer choosing lowest vs highest, a lock file simply eliminates all doubts no matter what resolution strategy is chosen.

That's why commnads like paket outdated exists though. You can totally show updates and that's not a fundamental problem. We just need to build more experiences around that. Frankly I think showing transitive updates is crazy but that's just me.

I think at this point we need to settle on the fact that we will never agree on this and that is OK, these two camps exist and it'd be nice if both can keep existing :+1: I'll drop the discussion on IDE files etcetera in the interest of staying on topic :tada:

Anyways, this is a philosophical difference we have and I'm not trying to convince you of anything.

This intrigues me :) What specifc set of changes do you see required for paket to play nicely and not shoehorn its way around DNX? Up until now the discussion has been around DNX providing hooks for paket to do its thing but this flips it around.

Leaving dependencies in project.json as an example. Also potentially declaring dependencies in global.json (this has been requested in the past). Does paket needs it's own file format? What if we can preserve some of the Paket ness without changing the file formats and tooling around that? It's a different tool that has a different algorithm for dependencies but respects the file format as is?

[eatdrinksleepcode]

There's only Paket and NuGet.

There's only Paket and Nuget today. In a healthy young ecosystem we should be seeing several more options at least in the near to medium term, with long term convergence towards a smaller number of options as the ecosystem matures and stabilizes. You should be doing everything you can to encourage this exploration.

That defeats the purpose. If you're not going to use the integrated experience there's no point going to this model.

There's no point in wanting the drastically simplified project management that project.json provides without being forced into the very narrow view of dependency management that Nuget holds? You're right, given the tight coupling that exists today, there is no point; but with just a little bit of looser coupling / extensibility, there would be.

NuGet/DNU chooses the lowest version.

Which is usually the wrong choice. Newer versions have new APIs or bug fixes that I may depend on; choosing the lowest version means those new APIs are unavailable and my build will fail, or worse it won't fail but I am missing some change in the newer version that I need/want. The better choice is to choose the highest version to make sure that all APIs and fixes are available, but WITHOUT changing major versions to avoid introducing breaking changes. If no version is available that satisfies that constraint, automatic dependency resolution should fail, and I should have to resolve it manually.

Unless you delete packages from your feed, transitive dependencies DO NOT CHANGE.

Packages get removed from feeds all the time for all sorts of reasons. With a lock file, a disappearing package will cause a build failure, which I can then resolve. With DNU, a new version is silently chosen, resulting in non-reproducible builds. This is not FUD; this is fact.

The reason we don't need this file for repeatable builds is because dependencies don't float.

This is flat out wrong, for the reasons that I and many other have stated. If you don't lock your dependencies to exact versions, you cannot guarantee repeatable builds. Period.

Java has a bunch of IDE specific files when you use eclipse with different types of projects.

Yes, there are IDE specific files for the IDE. The build doesn't need or care or even know about any of them. Do you not understand the enormous value that has?

Many developers from the community have come here to express their concerns, explain why the solutions you have designed (mostly in isolation) do not meet their needs, and try to work with you, not only to meet their needs but to expand the community as well. You have responded by being defensive, accusing people of spreading FUD, expressing bewilderment that anyone would gasp want to use a tool other than than what Microsoft has produced, and generally doing everything except listen to the very people you are supposedly designing these solutions for. Your tone has been arrogant, insulting, and indicative of the very closed-minded attitude that Microsoft has spent so much effort trying to convince the community doesn't exist anymore. As someone who has championed .NET for years and has spent the last year telling people who have long since given up on Microsoft that now is the time to give it another chance, watching this "conversation" (and the others related to it) has gone from upsetting to infuriating to defeating. What am I supposed to tell people who read these conversations and see Microsoft so stalwartly committed to defending the choices they have made behind closed doors, regardless of any argument or evidence, instead of engaging with and learning from - yes, learning from - the community that is their lifeblood?

I feel like so much more needs to be said, but I don't have the words...

[Mpdreamz]

I'd say lean on floating versions for your own dependencies (things that you control) and for 3rd party deps, just use the command that updates things.

Missing the point completely, you still have no repeatable builds for the floating versions you control. Using the commands is still per project this also needs fixing.

That's why comnads like paket outdated exists though. You can totally show updates and that's not a fundamental problem.

Sure but I'm more talking about greenfield deps. with paket I simply state the nuget id and then where I want to reference it, paket install and I am done. I never had to care about the version. Same goes for transitive deps, if its a newly introduced transitive dependency with a [6. range i'd like it to pick the latest 6.* release out of the box. I do not want to run a manual update command after. The fact nuget does NOT do this silently hurts nuget package authors IMO.

Frankly I think showing transitive updates is crazy but that's just me.

It's just you because you have no lock file to go back to :D

Anyways, this is a philosophical difference we have and I'm not trying to convince you of anything.

Not a fair statement, I am not the one in a particular position that needs convincing. For what its worth neither am I though, i just want to keep stressing choice is good.

Leaving dependencies in project.json as an example. Also potentially declaring dependencies in global.json (this has been requested in the past). Does paket needs it's own file format? What if we can preserve some of the Paket ness without changing the file formats and tooling around that? It's a different tool that has a different algorithm for dependencies but respects the file format as is?

I don't want to speak for @forki but I think this was the first approach he explored. From a usability perspective it'll be unpleasant though since VS will kick off restores differently then paket would do.

I would love to hear why you feel this proposed solution doesn't gel.

[forki]

If we're getting down the facts here are the reason it's FUD: NuGet/DNU chooses the lowest version. Unless you delete packages from your feed, transitive dependencies DO NOT CHANGE.

That's not correct. Here are 3 scenarios:

• Just let someone unlist a single transitive dependency and basically your complete depedency graph can change because of changing rquirements.
• Nuget fixs bug in resolution algorithm ==> new resolution of transitive deps
• There is <add key="dependencyVersion" value="Highest" /> in the config which makes resolution basically machine dependend. If your woworker/CI/deploy machine has wrong config you will have a bad time.

It's like me saying that Paket should prefer lower versions

http://fsprojects.github.io/Paket/nuget-dependencies.html#Strategy-modifiers

[davidfowl]

Which is usually the wrong choice. Newer versions have new APIs or bug fixes that I may depend on; choosing the lowest version means those new APIs are unavailable and my build will fail, or worse it won't fail but I am missing some change in the newer version that I need/want. The better choice is to choose the highest version to make sure that all APIs and fixes are available, but WITHOUT changing major versions to avoid introducing breaking changes. If no version is available that satisfies that constraint, automatic dependency resolution should fail, and I should have to resolve it manually.

Agree to disagree. There's no point arguing this. It's something we'll NEVER see eye to eye on. I respect each of the tools does things differently for different reasons.

This is flat out wrong, for the reasons that I and many other have stated. If you don't lock your dependencies to exact versions, you cannot guarantee repeatable builds. Period.

See my above answer.

Packages get removed from feeds all the time for all sorts of reasons. With a lock file, a disappearing package will cause a build failure, which I can then resolve. With DNU, a new version is silently chosen, resulting in non-reproducible builds. This is not FUD; this is fact.

Nothing solves that besides an artifact repository, your own nuget feed or checking in the packages. A file with a list of versions can't repeat a build if the packages don't exist in the feed.

Yes, there are IDE specific files for the IDE. The build doesn't need or care or even know about any of them. Do you not understand the enormous value that has?

Who said the build needs them. I specifically called out msbuild interop as a reason for having those files. Also as a project system existing in VS, it's also required for solution builds inside of that tool.

Many developers from the community have come here to express their concerns, explain why the solutions you have designed (mostly in isolation) do not meet their needs, and try to work with you, not only to meet their needs but to expand the community as well. You have responded by being defensive, accusing people of spreading FUD, expressing bewilderment that anyone would gasp want to use a tool other than than what Microsoft has produced, and generally doing everything except listen to the very people you are supposedly designing these solutions for. Your tone has been arrogant, insulting, and indicative of the very closed-minded attitude that Microsoft has spent so much effort trying to convince the community doesn't exist anymore. As someone who has championed .NET for years and has spent the last year telling people who have long since given up on Microsoft that now is the time to give it another chance, watching this "conversation" (and the others related to it) has gone from upsetting to infuriating to defeating. What am I supposed to tell people who read these conversations and see Microsoft so stalwartly committed to defending the choices they have made behind closed doors, regardless of any argument or evidence, instead of engaging with and learning from - yes, learning from - the community that is their lifeblood?

I feel like so much more needs to be said, but I don't have the words...

I'm sorry you feel that way. I'm not trying to be closed minded at all, I've even used paket myself before and during the development of the DNX project system. What I've been trying extremely hard to do is explain the how and why things work the way they do without emotion in a way that just discusses the facts. I've even said that we'd be willing to look at solution that potentially would require changes to Paket so that it fits more with the other solution.

I actually don't think things are as bad as people are making it seem.

[davidfowl]

Just let someone unlist a single transitive dependency and basically your complete depedency graph can change because of changing rquirements.

Unlisting doesn't affect nuget3 or dnu's dependency resolution behavior. It did with NuGet v2 it doesn't with v3.

Nuget fixs bug in resolution algorithm ==> new resolution of transitive deps There is in the config which makes resolution basically machine dependend. If your woworker/CI/deploy machine has wrong config you will have a bad time.

That feature does exist (unfortunately) and I agree to make it work "well" you'd need a lock file or sorts. I think that only proves my point though.

[forki]

Unlisting doesn't affect nuget3 or dnu's dependency resolution behavior.

What? ;-) Don't you think this is a bug in the resolver? If I unlist a package I want people to not use it. But I want still reproducible builds. That's why it was invented and NuGet didn't just delete the packages.

[forki]

anyways.

Does paket needs it's own file format? What if we can preserve some of the Paket ness without changing the file formats and tooling around that?

I don't think Paket needs it's own file format just to have it's own file format. Actually I hope we can settle on something that work for everybody. But having versions in the project.lock is very unfortunate for use. The main reason is that we change these versions a lot during updates and we can't gitignore a single block in a file. Paket always tries to have a global view so it doesn't make sense to manage dependency versions on te project level (we would introduce the possibility of version conflicts). That's why I found the snapshot file idea really nice. This would allow us to generate that file and gitignore it.

That's basically the whole issue. I could already have started and generate the dependencies block, but it doesn't feel right. Also generating the project.lock.json doesn't feel right since you already have working (and changing) tooling for that.

[robertmuehsig]

Maybe just keep in mind: project.json is not the only way of building libraries/dnx stuff. csproj stuff will stay, so keep calm.

[isaacabraham]

@GerjanOnline From a consumer point of view, Paket has as much to do with F# as JSON .NET has to do with C#, if not less i.e. yes it is written in F# - and is used by a lot of the F# community - but it works just as well for C# and VB .NET projects. So I think the issues here are not about the F# language or community as such, as Paket is used by e.g. C# developers as well.

I agree though the atmosphere is unpleasant and not at all what I'm used it in my experiences of working in open source projects in the .NET space.

[GerjanOnline]

@robertmuehsig @isaacabraham Got it, sorry for that. I just hope I can use Paket for my future projects like asp.net 5/DNX and I'm seeing a lot of negative comments/tension

[forki]

I'm not happy with how things escalated here. Now things are described as a F# community issues or a Paket community issue. That's not good since we designed Paket as a solution for all .NET projects and always wanted to create a nice community around it. I was hoping to find a technical solution to enable Paket for DNX projects. Nothing else.

Until things have clamed down I close this and #736 as "won't fix".

[SamB]

Hmm, I only meant to refer to "project.json the package.config replacement" (as I've seen used in msbuild and rosyln's source trees, and as @davidfowl says UWP projects use, with .csproj or .vbproj files that mainly differ from the typical in that they do contain explicit references to assemblies from the packages that they depend on), not "project.json the project format".

And I certainly didn't mean to start a flame war.

(Incidentally, would anyone happen to know where the "NuGet.MsBuild.Integration" package comes from?)

[edblackburn]

I have a C# .NET service. It uses paket, which I have been very, very happy with. I just googled paket CoreClr to investigate what is required to port my service to CoreClr and I have to say this thread is thoroughly depressing and aggressive in tone.

This thread is a massive, massive turn off to CoreClr and to .NET OSS / ecosystem in general. I really hope this is a temporary blip and not a mask slipping.

Frankly it's embarrasing to read.

[CumpsD]

Have things improved over the last 4 months?

[forki]

no comment

[HoraceGonzalez]

Does anything change now that project.json is out of the picture?

[forki]

That's not clear yet. But maybe we got a new chance for cooperation. On May 20, 2016 9:38 PM, "HoraceGonzalez" notifications@github.com wrote:

Does anything change now that project.json is out of the picture?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/fsprojects/Paket/issues/1230#issuecomment-220699599

[smoothdeveloper]

@HoraceGonzalez to answer "how it stacks up against NuGet.exe", see https://github.com/dotnet/cli/issues/3123 which is an "indepedantly" funded investigation on the matter :smile: