nuget pack ignores package.config dependencies

[kkm000]

I cannot get nuget pack X.csproj to recognize package dependencies in a project. Amazingly, when packaging, the diagnostic message “Found packages.config. Using packages listed as dependencies” is printed, but in the end the <dependencies/> tag in the .nuspec file inside the package is empty.

The packages.config for the project does indeed contain references:

<packages>
  <package id="SmartAction.Logger" version="1.0.2.0" targetFramework="net40" />
  <package id="SmartAction.Pervasive" version="1.0.1.0" targetFramework="net40" />
</packages>

To narrow the problem down, I removed my own parallel .nuspec file, and mostly all switches from the nuget pack command:

> nuget pack libToneDetection.csproj -prop Configuration=Release

MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
Attempting to build package from 'libToneDetection.csproj'.
Packing files from '[snip]\Core\ToneDetection\libToneDetection\bin\Release'.
Found packages.config. Using packages listed as dependencies
Successfully created package '[snip]\Core\ToneDetection\libToneDetection\SmartAction.Audio.ToneDetection.1.0.0.0.nupkg'.

NuGet Version: 3.3.0.212

The only difference I can spot with this project is that its name is different from package name (I am trying to maintain them in sync but this is older stuff I am repackaging).

When I copied the project out of the solution with other projects to a temporary directory and rebuilt the package from there. Now one of the two dependencies from packages.config was added to the package:

<dependencies>
  <dependency id="SmartAction.Logger" version="1.0.2.0" />
</dependencies>

Thinking of differences between the two, the SmartAction.Logger package depends on SmartAction.Pervasive. But the package I am compiling really uses both.

To me, either behavior looks incorrect. Am I hitting a nuget bug, or is it a very cryptic packaging rule?

Xref: initially posted as http://stackoverflow.com/q/34421409/1149924

[yishaigalatzer]

if SmartAction.Logger depends on SmartAction.Pervasive it makes sense not to add both of them as direct dependencies. When you install the package dependencies are resolved transitively.

So I would look at what is the difference between the project in the solution and outside the solution. Is there something else going on there?

[kkm000]

@yishaigalatzer: Regarding transitivity, I heartily disagree. In C++, we had a very god rule that paid off many times: IWYU, "include what you use," instead of relying on some magically obtained knowledge that X includes Y, therefore I do not need to include Y even if I use it. Many cryptic errors bit those who did not follow the rule. And this is sans versioning.

Also, this non-inclusion breaks documented NuGet rules of package version resolution. Suppose I have a program P that depends on packages X (version not important) and Y. X also depends on Y. Now think of all different versioning rules that P and X can separately declare. If you slash the explicitly defined dependency of P on Y, all the dependency hell breaks loose. In out example case, P depends on Logger ≥1.0.2 and Pervasive≥1.0.1. If at some point the library P is resolved to use Logger=1.5.0 (by way of it's dependee, which declares, e. g., Logger≥1.5.0), which in turn depends on Pervasive≥1.0.0, out dependency on Pervasive≥1.0.1 is broken, as we'll get Pervasive=1.0.0 with the default resolve-minimal-version strategy. Also, if Logger=1.5.0 does not depend on Pervasive any longer (and this is more common scenario in the wild than lowering the dependency version), we will not get the Pervasive package libraries at all!

I think there are cases possible when a dependency can be dropped, but this would probably require building an inequality theorem prover (and, tongue in cheek, a Prolog interpreter to run it) into NuGet client. I beg you not to try to hack around this astonishingly complex issue, and fix NuGet to explicitly add the package dependencies and follow its documented behavior--otherwise we'll build just another version of DLL hell with NuGet.

I did not look further into the second part.I can look at NuGet's code or debug it, but not this week.

[yishaigalatzer]

I see your point, but making both dependencies declared, do not actually resolve the problem (it might solve the dependency dropped issue) but the install process picks up dependencies anyways and does the uber complex gyration of figuring out the right dependency graph. That is the heart of nuget, and not a hack.

If you want to be that explicit, I would not rely on packing csproj directly and instead pack from a nuspec, packing from csproj should be considered a very limited scenario and not uber "sophisticated". We will gladly consider PRs in this area, but the typical answer is that if you truly care about the exact results, consider using generating a nuspec. The key problem here, is that it is not clear from the definition of the packages.config if as a user you chose to depend on all the dependencies directly or if they are an implementation detail. Lifting the dependencies to be top-level make cause the exact reverse effect of including packages a user no longer require and might even break.

With the move to project.json, where the dependencies are declared transitively upfront and never flattened, we would be able to do better. And will leave the decision of what to depend on to the end-user (aka you :) ) So if you want to depend on a common dependency, you can do that, and if you consider it an implementation detail of the package you depended on than you don't.

I hope this all makes sense, even if it is a bit frustrating.

[kkm000]

I see. So the actual problem here is that the csproj and its packages.config together have lost information why the dependency is in packages.config, so you cannot distinguish those added by the user from those included as package dependencies. That's a big deal, a nasty problem indeed! The whole infrastructure, including Visual Studio plugins, needs to be fixed to retain this bit of information.

I do not even know what "project.json" is. Visual Studio still uses MSBuild as of 2015. "You" are very far ahead of "us", sorry :)

[yishaigalatzer]

project.json is a mechanism that also works with msbuild, in fact nuget is build this way. NuGet 3.1 introduced it for UWP apps, and it provides quite a few advantages (even if it is not officially supported 100% yet).

Here is a quick primer on the subject and conversion, you can also look at our docs for that.

[kkm000]

Very interesting, thanks! So will it work with regular library csproj projects that do not do anything on install (basically, just packaged from the project file by default)?

Are "frameworks" and "runtimes" clauses required?

[yishaigalatzer]

regular library csproj projects that do not do anything on install (basically, just packaged from the project file by default)?

Can you elaborate?

frameworks yes - required so we pick the right target framework from the pacakges

runtimes required for copying assets to output, but not for compilation. We put it everywhere, but you only really require it for unit tests and console apps (or end user apps)

key limitation Doesn't work well for packages bringing content for web applications like jquery

[kkm000]

Thank you for the explanation. I am wondering is it technically possible/feasible to default framework to the one targeted by the current build? This is interesting, I have a few multi-target low level libraries that have separate configurations for net40 and net45. I had to manually make HintPath conditional on the Configuration variable in these when they are built as multitarget packages as well. If it was possible to automagically sensibly use the right library in the current build configuration, that would be a big boon indeed!

Can you elaborate?

Most of our libraries are just DLLs you would build by starting off the normal "C#/F# library" template, throwing in some .cs/.fs files and compiling. They are packaged as simply as running "nuget pack" without a nuspec. Of more complicated cases are multi-target libraries (net40, net45), and libraries with Code Contract dlls. None have .ps1 scripts; also, I do not think we ever deploy any content files at all.

[kkm000]

Ah, and there are also some packages that extend MSBuild, with .targets. .props and MSBuild tasks DLLs.

[yishaigalatzer]

So specifying multiple target frameworks is not yet supported with csproj, but works fine with raw project.json including automatically generating multiple builds and nuget packages (see ASP.NET class library in Update I). That unfortunately doesn't support msbuild targets of F# just yet.

You can use msbuild targets from nuget packages with project.json but it doesn't support multi target compilation and you will have to alter the target framework for now. We are working on changing that in the next release

[kkm000]

Thanks! I'll certainly give it a try!

[yishaigalatzer]

Closing for now unless we have a further action to take here (feel free to re-ping)

[kadamgreene]

This issue is happening again with NuGet 3.5.0.1365. It says its inspecting the packages.config, but nothing happens. This is what I'm doing:

1. I have created a solution (Libraries.sln)
2. I added a project to it (Utilitiies\Utilities.csproj).
3. I add some packages to it from nuget (NHibernate 4.0.4 to be precise)
4. From the \Utilities folder, I run "nuget pack". It says its using packages.config to discover dependencies, but then I get in the resulting nupkg file.

I have tried the suggestion of putting the nuget.config in the solutions folder (pointing to the packages folder) and again in the Utilities folder (pointing to ..\packages), but to no avail. What am I doing wrong? What is wrong?

[ashelley]

I'm having this same problem...

NuGet Version: 3.4.4.1321

packages.config

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.Owin" version="3.0.1" targetFramework="net461" />
  <package id="Microsoft.Owin.Hosting" version="3.0.1" targetFramework="net461" />
  <package id="Microsoft.Owin.Testing" version="3.0.1" targetFramework="net461" />
  <package id="Newtonsoft.Json" version="9.0.1" targetFramework="net461" />
  <package id="Owin" version="1.0" targetFramework="net461" />
  <package id="Owin.Builder" version="0.8.5" targetFramework="net461" />
  <package id="System.Data.SqlLocalDb" version="1.15.0" targetFramework="net461" />
  <package id="xunit" version="2.1.0" targetFramework="net461" />
  <package id="xunit.abstractions" version="2.0.0" targetFramework="net461" />
  <package id="xunit.assert" version="2.1.0" targetFramework="net461" />
  <package id="xunit.core" version="2.1.0" targetFramework="net461" />
  <package id="xunit.extensibility.core" version="2.1.0" targetFramework="net461" />
  <package id="xunit.extensibility.execution" version="2.1.0" targetFramework="net461" />
</packages>

Myproject.nuspec

<?xml version="1.0"?>
<package >
  <metadata>
    <id>$id$</id>
    <version>$version$</version>
    <title>$title$</title>
    <authors>Me</authors>
    <owners>$author$</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Project Description</description>
    <releaseNotes>Project Release Notes</releaseNotes>
    <copyright>Copyright 2016</copyright>
    <tags>XUnit Test Extensions</tags>
  </metadata>
  <files>
    <file src="NuGet\app.config.install.xdt" target="content" />
    <file src="Asserts\**\*.cs" target="Content\Asserts" />
  </files>
</package>

building from cs project file:

Attempting to build package from 'MyProject.csproj'.
MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
Building project 'C:\inetpub\wwwroot\MyProject\src\MyProject.csproj' for target framework '.NETFramework,Version=v4.6.1'.
__________________________________________________
Project "C:\inetpub\wwwroot\MyProject\src\MyProject.csproj" (default targets):

Properties\AssemblyInfo.cs(36,32): warning CS7035: The specified version string does not conform to the recommended format - major.minor.build.revision
Done building project "MyProject.csproj".
Packing files from 'C:\inetpub\wwwroot\MyProject\src\bin\Debug'.
Using 'MyProject.nuspec' for metadata.
Found packages.config. Using packages listed as dependencies
Successfully created package 'C:\inetpub\wwwroot\MyProject\build\out\MyProject.0.0.2.nupkg'

But the dependencies are not added to the .nupkg. Do I have to manually create dependency element some how in nuspec file?

@yishaigalatzer reping is this still an issue?

EDIT: xrefing this issue here: https://github.com/NuGet/Home/issues/1441

I'll see if i can come up with a minimum test case

[yishaigalatzer]

Yes, please do and we can re-open.

Please do two things first just for leveling the playing field.

Get nuget 3.5 and make sure all the packages are restored.

[ashelley]

@yishaigalatzer I have created a minimum repro of the problem here:

https://github.com/ashelley/NugetDepsIgnored-Repro

I think it has something to do with the project layout but not sure. I have created "Good", a project that generates the expected results and "Bad": A project that does not add node properly to nupkg/.nuspec file.

Steps to reproduce:

1) clone above repository and change into the directory you cloned 2) open both solutions in visual studio 2015 and build them to ensure packages get restored 3) create a build artifact directory 4) build each .nupkg

Adam@DEVELOPER-03 MINGW64 /c/inetpub/wwwroot
**$ git clone https://github.com/ashelley/NugetDepsIgnored-Repro nuget-repro**
Cloning into 'nuget-repro'...
remote: Counting objects: 25, done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 25 (delta 6), reused 25 (delta 6), pack-reused 0
Unpacking objects: 100% (25/25), done.
Checking connectivity... done.

Adam@DEVELOPER-03 MINGW64 /c/inetpub/wwwroot
**$ cd nuget-repro/**

Adam@DEVELOPER-03 MINGW64 /c/inetpub/wwwroot/nuget-repro (master)
**$ mkdir build**

**$ nuget pack ./Good/src/NugetDepsIgnored-Repro.csproj -Build -OutputDirectory ./build**
Attempting to build package from 'NugetDepsIgnored-Repro.csproj'.
MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
Building project 'C:\inetpub\wwwroot\nuget-repro\Good\src\NugetDepsIgnored-Repro.csproj' for target framework '.NETFramework,Version=v4.6.1'.
Packing files from 'C:\inetpub\wwwroot\nuget-repro\Good\src\bin\Debug'.
**Found packages.config. Using packages listed as dependencies**
WARNING: Description was not specified. Using 'Description'.
WARNING: Author was not specified. Using 'Adam'.
Successfully created package './build\NugetDepsIgnored-Repro.1.0.0.0.nupkg'.

Adam@DEVELOPER-03 MINGW64 /c/inetpub/wwwroot/nuget-repro (master)
**$ nuget pack ./Bad/src/NugetDespIgnored-Repro2.csproj -Build -OutputDirectory ./build**
Attempting to build package from 'NugetDespIgnored-Repro2.csproj'.
MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
Building project 'C:\inetpub\wwwroot\nuget-repro\Bad\src\NugetDespIgnored-Repro2.csproj' for target framework '.NETFramework,Version=v4.6.1'.
Packing files from 'C:\inetpub\wwwroot\nuget-repro\Bad\src\bin\Debug'.
**Found packages.config. Using packages listed as dependencies**
WARNING: Description was not specified. Using 'Description'.
WARNING: Author was not specified. Using 'Adam'.
Successfully created package './build\NugetDespIgnored-Repro2.1.0.0.0.nupkg'.

Note: in both good/bad situations the nupkg compiler has found and detected the packages.config files.

5) After creating the nupkg files extract them an inspect the .nuspec dependency node in each package

Expected results: Each nuspec file in the packages should have the dependency nodes added.

Actual results: After inspecting the dependency nodes in each nuspec file only one has the expected results

Adam@DEVELOPER-03 MINGW64 /c/inetpub/wwwroot/nuget-repro (master)
$ grep -i 'dep' ./build/NugetDepsIgnored-Repro.1.0.0.0/NugetDepsIgnored-Repro.nuspec
    <id>NugetDepsIgnored-Repro</id>
    <title>NugetDepsIgnored-Repro</title>
    <dependencies>
      <dependency id="System.Data.SqlLocalDb" version="1.15.0" />
      <dependency id="xunit" version="2.1.0" />
    </dependencies>

Adam@DEVELOPER-03 MINGW64 /c/inetpub/wwwroot/nuget-repro (master)
$ grep -i 'dep' ./build/NugetDespIgnored-Repro2.1.0.0.0/NugetDespIgnored-Repro2.nuspec
    <dependencies />

Note: We can see here that the dependency nodes were not added to the nupkg file in the "Bad" project

EDIT: @yishaigalatzer Will retry with 3.5 sorry... this was done with version noted in my previous comment

[ashelley]

Okay nuget 3.5 fails to create the package at all:

NuGet Version: 3.5.0.1938

$ nuget pack ./Bad/src/NugetDespIgnored-Repro2.csproj -Build -OutputDirectory ./build
Attempting to build package from 'NugetDespIgnored-Repro2.csproj'.
MSBuild auto-detection: using msbuild version '14.0' from 'C:\Program Files (x86)\MSBuild\14.0\bin'.
Building project 'C:\inetpub\wwwroot\nuget-repro\Bad\src\NugetDespIgnored-Repro2.csproj' for target framework '.NETFramework,Version=v4.6.1'.
Microsoft (R) Build Engine version 14.0.25420.1
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 11/01/2016 2:57:15 PM.
Project "C:\inetpub\wwwroot\nuget-repro\Bad\src\NugetDespIgnored-Repro2.csproj" on node 1 (default targets).
GenerateTargetFrameworkMonikerAttribute:
Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output files are up-to-date with respect to the input files.
CoreCompile:
Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
CopyFilesToOutputDirectory:
  NugetDespIgnored-Repro2 -> C:\inetpub\wwwroot\nuget-repro\Bad\src\bin\Debug\NugetDespIgnored-Repro2.dll
Done Building Project "C:\inetpub\wwwroot\nuget-repro\Bad\src\NugetDespIgnored-Repro2.csproj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.15
Packing files from 'C:\inetpub\wwwroot\nuget-repro\Bad\src\bin\Debug'.
Found packages.config. Using packages listed as dependencies
Failed to retrieve information from remote source 'C:\inetpub\wwwroot\nuget-repro\Bad\src\packages'.

The packages are restored so I'm not sure if this is related to this:

https://github.com/NuGet/Home/issues/3343

Still seems like a bug but at least the failure is loud instead of silent.

EDIT:

possibly related to this too:

https://github.com/NuGet/Home/issues/3058

The situation is a bit different though:

nuget in visual studio installs the package cache along side the solution file however nuget pack doesn't know how to locate this directory it seems.

[ashelley]

I've doubled checked nuget's help file on its pack command:

$ nuget pack /?
usage: NuGet pack <nuspec | project> [options]

Creates a NuGet package based on the specified nuspec or project file.

     Specify the location of the nuspec or project file to create a package.

options:

 -OutputDirectory                     Specifies the directory for the created NuGet package file. If not specified, uses the current directory.
 -BasePath                            The base path of the files defined in the nuspec file.
 -Version                             Overrides the version number from the nuspec file.
 -Suffix                              Appends a pre-release suffix to the internally generated version number.
 -Exclude +                           Specifies one or more wildcard patterns to exclude when creating a package.
 -Symbols                             Determines if a package containing sources and symbols should be created. When specified with a nuspec, creates a reg
                                      ular NuGet package file and the corresponding symbols package.
 -Tool                                Determines if the output files of the project should be in the tool folder.
 -Build                               Determines if the project should be built before building the package.
 -NoDefaultExcludes                   Prevent default exclusion of NuGet package files and files and folders starting with a dot e.g. .svn.
 -NoPackageAnalysis                   Specify if the command should not run package analysis after building the package.
 -ExcludeEmptyDirectories             Prevent inclusion of empty directories when building the package.
 -IncludeReferencedProjects           Include referenced projects either as dependencies or as part of the package.
 -Properties +                        Provides the ability to specify a semicolon ";" delimited list of properties when creating a package.
 -MinClientVersion                    Set the minClientVersion attribute for the created package.
 -MSBuildVersion                      Specifies the version of MSBuild to be used with this command. Supported values are 4, 12, 14. By default the MSBuild
                                      in your path is picked, otherwise it defaults to the highest installed version of MSBuild.
 -Help                           (?)  help
 -Verbosity                           Display this amount of details in the output: normal, quiet, detailed.
 -NonInteractive                      Do not prompt for user input or confirmations.
 -ForceEnglishOutput                  Forces the application to run using an invariant, English-based culture.

There doesn't seem to be a way to provide context for where the packages folder might be however visual studio uses it's own solution file to provide this context. Maybe there should be an explicit way to specify where nuget pack can look for packages?