A possible way forward for Content in Transitive Restore

[analogrelay]

Currently, the project.json/transitive restore workflow does not support content/ files. This hasn't been a problem in DNX because DNX is focused on Web scenarios and tools like bower and gulp/grunt are both capable and well-understood for solving this problem. Unfortunately, in other non-Web scenarios, these are less ideal (though they would, in fact, work fine).

Also, the bower/gulp/grunt workflow does not cover another major scenario for Content files in NuGet packages: Configuration to go along with .NET components. Often NuGet packages will include example configuration files to go along with their components. (Note: I am intentionally not including packages with content transforms, like XDT or Xml Merge, in this list; those still need to be considered later)

To that end, there is a possible workflow to make content files work relatively well in the transitive restore world. Currently, when a package containing content is restored using the transitive restore system, the content files are placed on disk, within the machine-level package installation directory (i.e. ~/.nuget/packages). However, there is no process to integrate them into the project files themselves.

I propose a new command/gesture: nuget install-content (or perhaps even just nuget install since this could extend beyond content in theory). This command takes the packages defined in project.lock.json (thus it must happen AFTER a restore) and applies any project-level transformations defined in those packages. Much like how the DNX runtime behaves, the process would check that the lock-file is up-to-date relative to the project and fail if it is out-of-date (informing the user that they must restore first).

During installation of a package via the Visual Studio NuGet GUI, NuGet will run the following process (bold steps are new in this proposal)

1. Modify the project.json to include the new package
2. Execute nuget restore to update the lock file
3. Execute nuget install-content to update/install content files (possibly constrained to only new packages?)

As a result of this, installation of a package through Visual Studio will include content

The install-content command should be idempotent (that is, multiple invocations should have the same effect as a single invocation) and will be best-effort (i.e. no rollback; if a content file fails to install, the user is informed and other files are installed). If the destination file for a content file exists, it is skipped by default (the user may have the option to override this behavior via options/switches on install-content).

In theory, content transformations could also be supported here, but I'm putting that out of scope for the moment.

I think this proposal integrates relatively well with the declarative nature of the new transitive restore model by basically leaving it alone. Instead, we add the ability to execute an imperative process when the user needs it. It also takes NuGet out of the business of trying to fiddle with conflicting content during restore. The install-content command can have options for managing conflicts as needed, and since it is intended to be an interactive process, it can have a few more quirks than the restore process which is designed to be entirely non-interactive

There is of course a risk of creating a higher level of complexity. I'm certainly open for ideas to mitigate that complexity :).

/cc @emgarten @yishaigalatzer @davidfowl

[yishaigalatzer]

Alternatively we might want to use the shared source, and inclusion of content files by the project system

[ljw1004]

Here's how I see it. When I add a NuGetv3 package reference, this causes at build-time a new item to be added to the C# compiler command line: "/r:a.dll"

Likewise, when I add a NuGetv3 package reference to something like sqlite-net, I think this should cause at build-time a new item to be added to the C# compiler command line: "a.cs". Just like the reference case.

More precisely: there is a large class of NuGet work whose behavior is that, at build-time, it causes msbuild to dynamically generate transient entries that aren't written to disk. Some of them contain <References> items (which is what's supported today). Others can contain .cs files (like in sqlite-net). Others can contain .png files or other resources (which is what lots of control authors want to see).

I recognize that there are two completely different uses of content. One, like I described, is for when you want the content to be included in your build output. The other use of content, as in the "-install-content" suggestion at the top of this issue, is about actually adding artefacts into the user's own project that the user will now own and control and edit.

I conjecture that the first use (.cs files, .dll files, .png files...) is much more common in client scenarios, and the second use (content) might be more common in web scenarios.

[yishaigalatzer]

We want to move source code to a shared source model, so you can have a package with vb for vb projects and cs for cs projects. But your suggestion makes sense. In fact there is hardly any nuget work to make it happen. It's all the msbuild task, we just need to build a compelling story and not get caught by the legacy burden.

We should look at moving forward, not backwards

[analogrelay]

I agree with @ljw1004 's general statement at the end. Web scenarios, which is where NuGet originated, tend to need ways to edit files in the user's project (though to be perfectly clear, I mean ASP.NET 4.x scenarios, since ASP.NET 5 does not allow this). However, both ASP.NET 5 and client app scenarios are moving towards the model of immutable content provided by packages. I definitely think it's a good idea to focus on that new model.

However, there are going to be projects that depend on content transformation for a very long time. ASP.NET 4.x is not going away, nor is web.config. This way forward is more about ensuring there is at least some user gesture to make that behavior work. Immutable content and shared-source packages are new features that can be used by new components, but I this issue was created explicitly to look at how we can handle the legacy cases without burdening ourselves (by adding them to the primary package restore process).

[kevinkuszyk]

For me the lack content transforms for adding default a config.cs to the consuming project is a deal breaker. Many libraries rely on this feature to bootstrap default configuration.

I like the direction NuGet is taking in terms of focusing on delivering assemblies, and leaving client side content to package mangers like bower, but please bring back this feature.

[yishaigalatzer]

Related discussion - https://github.com/NuGet/Home/issues/1020

[o-o00o-o]

+1 for content to be revived

We currently use Content to allow distributing .targets and .props files for inclusion into project file via elements. This is necessary as VS needs to know where these files are in advance (you can have a property at root level that defines this, but this ultimately needs to be statically defined).

We also use it to allow including other referred scripts (using :r sqlcmd statements) again which requires a known in advance location.

We either would need a way to update this known location in content/project files so they are pointing to the correct folder or I would need this content folder capability as described here and as exists in 2.8. In my case these are real "library" type content files rather than modified boilerplate types so it fits well

If anyone has another method to solve this I'd be happy to hear about it, but I'm glad that content appears to be coming back.

[analogrelay]

Note that .targets and .props files are still supported in project.json-based NuGet. See https://docs.nuget.org/create/uwp-create#msbuild-targets-and-props-files-in-packages

[o-o00o-o]

Yes I'm aware of this however sadly I am working with a project that doesn't support nuget (SSDT projects) and so to workaround this I create a sibling c# project to provide the nuget support and then refer across to the library type files from the main project. This works well with the Content creation model and I have to do the same thing with any dacpac references etc. so they can be updated automatically and easily across many consumers.

Not ideal but far better than any alternative I've found so far!

PS. I was considering writing an install.ps1/init.ps1 script to automate other parts of this to graft in those pieces from the project and by naming convention I could find the actual consumer project. Alas it seems that these ps1 scripts are no longer going to be supported so I haven't devoted any further time to it. Better of course would be to add ssdt support directly into Nuget but indications are that this isn't planned.

[MichaelKetting]

+1 for binging content back for CS-files, which I rely on to distributing utility classes, etc. +1 for xdt-support which are needed to setup configuration files of all kinds. For both, I like the option to use install-content / an automatic integrtion when using Visual Studio. It's only needed inside Visual Studio but it is needed there.

For web-content, in NuGet v2 I use install.ps1/uninstall.ps1 to add links to the content files to the project without actually copying them and to only add links if the nuget package is referenced by a web-client project, but not a web-library. I do this to a) support the Visual Studio Publish Web-Site feature and to avoid noise/re-including the web-content when NuGet-packing my web-library project.

I haven't actually looked at the newfangled web-content package managers since my eco-system relies on ten years of established conventions and code bases and you can't just drop in new concepts like that ;) It looks like project.json will support them out of the box, but what do you do with old-style projects? It certainly was easier when you just wrote a web control library and published it via NuGet.

It would certainly be interesting to have some sort of specification language that can register/unregister project items in projects. This would need to have project type filters, metadata, etc, of course.

[johncrim]

A couple problems with content files in NuGet 2:

• The NuGet project shouldn't "know" or force where the content files are located in the project. The classic example, which drives me nuts, is putting javascript files in the {{./Scripts/}} directory.
• Copying files from the nuget package to the target project has the implication (or accidental result) of adding nuget content files to the project's source control. Normally this is not a good thing, though there may be cases where it's desired.

In general the location of content files should be controlled by the project, and the project should be able to use the content files in the package directories without moving them.

What I would like to see is a really strong/helpful mechanism for the project build process to include content files from the package directories, without each build tool having to hard-code the paths. In general that means msbuild logic could easily access/manipulate the content files in the package directories. For web projects in particular, it would also be great for gulp files to have an easy way to pull the package content files into the gulp build logic.

[yishaigalatzer]

John - we totally agree. In fact we had another design meeting on this last night following very similar lines.

@emgarten I going to post design notes on the wiki

Sent from my Windows Phone

---

From: John Crimmailto:notifications@github.com Sent: ‎9/‎25/‎2015 6:35 AM To: NuGet/Homemailto:Home@noreply.github.com Cc: Yishai Galatzermailto:yigalatz@microsoft.com Subject: Re: [Home] A possible way forward for Content in Transitive Restore (#627)

A couple problems with content files in NuGet 2:

• The NuGet project shouldn't "know" or force where the content files are located in the project. The classic example, which drives me nuts, is putting javascript files in the {{./Scripts/}} directory.
• Copying files from the nuget package to the target project has the implication (or accidental result) of adding nuget content files to the project's source control. Normally this is not a good thing, though there may be cases where it's desired.

In general the location of content files should be controlled by the project, and the project should be able to use the content files in the package directories.

What I would like to see is a really strong/helpful mechanism for the project build process to include content files from the package directories, without each build tool having to hard-code the paths. In general that means msbuild logic could easily access/manipulate the content files in the package directories. For web projects in particular, it would also be great for gulp files to have an easy way to pull the package content files into the gulp build logic.

— Reply to this email directly or view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fgithub.com%2fNuGet%2fHome%2fissues%2f627%23issuecomment-143225727&data=01%7c01%7cyigalatz%40microsoft.com%7cae0540c98987449b434d08d2c5ae32ee%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=qXrJic2Bij2QcVdDg8t8t8jQ5VkAuAPvHH0napGAW4M%3d.

[o-o00o-o]

Yes, If the actual paths to the package folders could be made available via MSBuild properties at solution level (so available for every project in the solution regardless of the individual nuget support), then in my case I would be able to avoid having these as content.

My non-nuget supporting SSDT project would then be able to refer to the .targets and other files I need directly to the correct packages folder.

This would speed up installation / update time and solve a lot of these unnecessary content folders that packages have historically had to create at installation time.

[joeaudette]

I think content files that can be later edited is a very important and very common scenario in web apps.

For example creating a class library that has ViewComponents that need corresponding Views to be added to the consuming web app. This is a scenario I need and am currently blocked. http://stackoverflow.com/questions/31412210/how-to-include-views-in-asp-net-5-class-library-nuget-package

It seems as of beta8 and beta8 tooling with VS 2015 I can now package the views, however they are not installed into the web app that consumes the package. I guess that is currently a known issue that this thread topic is about. If I'm missing or misunderstanding some current way to do this please let me know.

For my needs I don't think I care that much if upgrading or re-installing a package would overwrite the content files (in my case view.cshtml files). My advice to consumers would be to copy the views with a new name before editing and then reference their copy of the view.

The suggested install-content command would fine with me as well.

The main thing is we really need something to support this scenario and I would like to think something is coming soon to support it. I have a viewcomponent for asp.net 5 navigation menus, breadcrumbs etc that is free and open source and I would like a convenient way to share it along with a set of example views.

[yishaigalatzer]

This is a related item we plan to address separately, the term is scaffolding content unlike this issue which covers immutable content.

[joeaudette]

Thanks for the clarification, is there a separate issue I can follow for scaffolding content so I can know when it is implemented?

[sylveon]

I need this!!! I can't start my project without the files from the sqlite-net nuget package, and I do not want to copy those files manually... Is there any way I can use the old system except using an old project? Ex: Editing something in the solution / project file

[yishaigalatzer]

We are shipping the solution content based packages next week

[sylveon]

@yishaigalatzer: What will happen with projects that installed content-based packages already but that NuGet didn't supported at this time? Will the content be added to the project, or it will be something more like @johncrim's proposal.

[yishaigalatzer]

It's a completely new model that packages have to opt into, so its orthogonal. The behavior is explained in our wiki and will go live on docs.nuget.org next week

[ljw1004]

Is it possible for a single package to work both under packages.config and also under the new behavior?

[sylveon]

@ljw1004 The latest nuget version supports two styles for backward compatibility. So I expect that it will still have backward compatibility, and just treat them the same as if they where installed using the new style, thus copying files as expected.

[yishaigalatzer]

Yes, it needs to author both folders. We plan to add a linking feature to help control the size of the packages

[sylveon]

Will nuget accept old unupdated packages with content or it is mandatory to use the new model for content to copy?

[yishaigalatzer]

Old packages will work in packages.config scenario only.

[sylveon]

How can I use this scenario on an Universal Windows app then?

[yishaigalatzer]

The package needs to support shared content model. If all you are looking is to use SQL lite grab the SQL lite uwp package

Sent from my Windows Phone

---

From: Charles Milettemailto:notifications@github.com Sent: ‎11/‎27/‎2015 12:51 PM To: NuGet/Homemailto:Home@noreply.github.com Cc: Yishai Galatzermailto:yigalatz@microsoft.com Subject: Re: [Home] A possible way forward for Content in Transitive Restore (#627)

How can I use this scenario on an Universal Windows app then?

— Reply to this email directly or view it on GitHubhttps://github.com/NuGet/Home/issues/627#issuecomment-160202104.

[sylveon]

V 3.3 is released but this issue isn't closed? 0_o

[yishaigalatzer]

Closed now - Content model is supported and documented. We will track separately work for scaffolded content https://github.com/NuGet/Home/issues/1792

[reyou]

Documentation at https://docs.nuget.org/Consume/ProjectJson-Intro should be updated since this issue is closed.

[yishaigalatzer]

Not sure what you mean, can you please elaborate? This issue was superseded by #1792

[reyou]

Maybe this is wrong place to request, but since there is no documentation edit/suggest at nuget.org, documentation over there assumes this issue is still open. But seems like it is closed already.

[yishaigalatzer]

Thanks! There is a way to suggest changes :) at the bottom of the page there is a link to an MD file, click it and you can submit a PR.

[reyou]

Ah, sorry to miss that :( Thanks for clarifying!