NuGet pack ignores dependencies in output nuspec for project.json

[rohit21agrawal]

This issue is being created out of the two issues filed under : https://github.com/NuGet/Home/issues/3075

This is tested against the the latest nightly build on the date of filing of issue. The behavior with 3.5.0-beta2 is very different in the sense that it does output the dependencies (in the output nuspec) but it completely ignores the provided nuspec file.

Repro steps:

1) Take a project.json based csproj (like UWP class library). 2) Add a nuspec like the following:

<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>ProjectA</id>
    <version>1.0.0</version>
    <authors>Rohit Agrawal</authors>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Helps convert packages.config to project.json</description>
  </metadata>
  <files>
    <file src="bin\Debug\*.dll" target="mylib" />
  </files>
</package>

3) Add Newtonsoft.json dependency in project.json 4) Run NuGet pack on the csproj.

Output nuspec is :

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
  <metadata>
    <id>ProjectA</id>
    <version>1.0.0</version>
    <authors>Rohit Agrawal</authors>
    <owners>Rohit Agrawal</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Helps convert packages.config to project.json</description>
    <dependencies />
  </metadata>
</package>

[rohit21agrawal]

@mistykuu Note that we have fixed the nuget pack case for csproj + project.json, however, it is expected behavior that the nuspec file will be completely ignored. anything you add in your nuspec file, can be added to your project.json file as well (reference: https://docs.microsoft.com/en-us/dotnet/articles/core/tools/project-json#packoptions)

In spite of this, if there is something you are still blocked on, please let us know and we'll track it accordingly.

[MistyKuu]

@rohit21agrawal I am not sure if I understand it correctly. What do you mean by "note we have fixed the nuget pack case for csproj + project.json" if in the next sentence you are saying that nuspec is ignored and it's expected behavior. So nuspec should be ignored and we should use project.json from now? Sorry maybe it's because I just woke up and I don't understand :)

Edit: I tried this and it doesn't produce anything in the package, owners are not used as well.

  "packOptions": {
    "owners": [ "Fabrikam", "Microsoft" ],
    "files": {
      "mappings": {
        "TextFile1.txt": "mylib/TextFile1.txt"
      }
    }
  }

[BladeWise]

@rohit21agrawal I knew that project.json could be used as a replacement for projectname.nuspec (it didn't seem to me that was going to be superseded by it), yet I have not found any documentation regarding tokens substitution. Trying to use this project.json

{
  "name": "Test",
  "version": "$version$",
  "title": "$title$",
  "authors": ["$author$"],
  "description": "$description$",
  "dependencies": {
    "Newtonsoft.Json": "9.0.1"
  },
  "frameworks": {
    "net451": {}
  },
  "runtimes": {
    "win": {}
  }
}

does not work, since project cannot be compiled: Error occurred while restoring NuGet packages: Error reading 'folder\project.json' at line 3 column 25 : '$version$' is not a valid version string. So I suppose token substitution is not available using project.json, which is quite a problem if I want to match package version with assembly version, expecially when using a tool to autmate version increment.

Edit: Using this project.json with 3.4.4.1321

{
  "name": "Test",
  "version": "1.0.0",
  "title": "Hello world",
  "authors": ["Me"],
  "description": "A desc",
  "dependencies": {
    "Newtonsoft.Json": "9.0.1"
  },
  "frameworks": {
    "net451": {}
  },
  "runtimes": {
    "win": {}
  }
}

produces the following nuspec inside the package

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2011/08/nuspec.xsd">
  <metadata>
    <id>ConsoleApplication6</id>
    <version>1.0.0</version>
    <title>ConsoleApplication6</title>
    <authors>TEST COMPANY</authors>
    <owners>TEST COMPANY</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Test description</description>
    <copyright>Copyright © TEST COMPANY 2016</copyright>
    <dependencies />
  </metadata>
</package>

whose information are taken directly from the assembly. Using 3.5.0.1520 (3.5.0-beta2) it produces this nuspec

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">
  <metadata>
    <id>ConsoleApplication6</id>
    <version>1.0.0</version>
    <title>Hello world</title>
    <authors>Me</authors>
    <owners>Me</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>A desc</description>
    <copyright>Copyright © TEST COMPANY 2016</copyright>
    <dependencies>
      <group targetFramework=".NETFramework4.5.1">
        <dependency id="Newtonsoft.Json" version="9.0.1" exclude="Build,Analyzers" />
      </group>
    </dependencies>
  </metadata>
</package>

which is better. Using the latest nightly build (3.5.1.1597) the same nuspec as 3.4.4.1321 is produced.

[MistyKuu]

It looks like packOptions are not used by nuget pack (I suppose they are used by dotnet pack). Can we expect the fix in 3.5 RTM to use either nuspec or project.json pack options?

[rohit21agrawal]

@BladeWise @MistyKuu there's a slight error due to which the nightly build is picking up builds from 3.5.1 branch (which is a special branch), however, the fix we made went into the dev branch. I am attaching a NuGet.exe as zip file here for you to try out.

UPDATE: The link to the nightly build has been fixed as well. It should give you a 3.5.0-rc package now.

NuGet.zip

[rohit21agrawal]

@BladeWise @MistyKuu to answer your other questions : project.json should be used as a replacement for nuspec in project.json based projects. If a project.json file exists, then NuGet will ignore the nuspec file. however, if you still want nuspec, you should pack using the nuspec file (note that you can pass nuspec file to the pack command).

We do support packOptions in project.json - the reason it's not working for you is because the build that you have does not include the fix I made, and project.json file is still being ignored in that build. Please try using the nuget.exe I have attached in my previous comment and let me know what issues you see there.

Thanks, Rohit

[MistyKuu]

@rohit21agrawal It doesn't seem to work. I can't see any changes. Could you please provide an example of packOptions that include files? Authors, owners etc are simply ignored unless you provide them in root of project.json.

[rohit21agrawal]

@MistyKuu Yes, we don't support tags like authors, owners under packOptions yet. (They will be supported in future releases, but I can't give you a clear time frame yet). However, the files tag does work and I have attached an example project for that.

ProjectA.zip

Here is the right way to use packOptions:

"packOptions": {
    "files": {
      "mappings": {
        "mylib/files": {
          "includeFiles": "./TextFile1.txt"
        }
      }
    }
  }

[MistyKuu]

@rohit21agrawal thank you very much., I confirm it's working.

[rohit21agrawal]

@MistyKuu glad to know that! As for your other concerns, here is a tracking issue:

https://github.com/NuGet/Home/issues/3161

[BladeWise]

@rohit21agrawal I can confirm that it works fine too, and I noticed that if I omit package info, the values from the assembly are used instead, which is fine. Now, I tested this version against #3128 (which I reported some days ago), but even this executable fails to generate a nuget package. I would really appreciate if you could have a look at the issue.

Edit: I tried to generate a package for a netstandard project, and the executable linked above still generates an invalid folder portable50. Moreover, what is the proper syntax to include files depending on wildcards? Previously in nuspec you could specify ./bin/Debug/**/*.txt to include all txt files in every bin\Debug subfolder, keeping the directory structure. Has this functionality been cut?

[MistyKuu]

Yup and what about MsBuild variables like $OutputPath$?

[rohit21agrawal]

@MistyKuu @BladeWise I haven't had a chance to try out if those things work or not, but if they aren't working for you, please feel free to file specific issues and we'll do our best to fix them.

[BladeWise]

@rohit21agrawal It does not work the way it worked with a nuspec, but honestly I cannot even say if it works or not. The current documentation regarding project.json is at best lacking... I don't even understand what is the difference between includeand includeFiles, and does not state if wildcards are allowed or not. Can you point me to a better documentation, or at least let me know what should be the proper syntax to include files matching this pattern /bin/Debug/**/*.txt?

Edit: Upon reading the docs again and again, it starts to make sense... I am performing more tests regarding packOptions, files and mappings and will reprt a separate issue if needed.

Regarding the issue at hand, I can confirm that latest nighly build produces proper dependencies.

[MistyKuu]

@rohit21agrawal the basic functionality doesn't work with packOptions. There are two main issues:

Packing directories ( this is connected with the second one because we can use wildcards to do that)

"packOptions": {
    "files": {
      "mappings": {
        "mylib/files": "./bin/Debug"
      }
    }
  }

Using wildcards

"packOptions": {
    "files": {
      "mappings": {
        "mylib/files": "*.pdb"
      }
    }
  }

And MsBuild variables...

These are really common scenarios - getting dependencies, adding files etc. With nuspec it worked flawlessly. If you are planning to obsolete nuspec please at least give us basic functionality in project.json.

[BladeWise]

@MistyKuu After a bit of testing I found some oddities with globbing patterns (as they are called in the docs), some not clearly document behaviors, and what I think is a bug. I can confirm that this

"packOptions": {
    "files": {
      "mappings": {
        "mylib/files": "*.pdb"
      }
    }
  }

works as intended, but refer to #3173 for further discussion regarding mappings.

@rohit21agrawal As you requested, a new issue (#3173) has been created.

[MistyKuu]

@BladeWise Ok it does work. Try this:

 "packOptions": {
    "files": {
      "mappings": {
        "mylib": {
          "include": [
            "bin/Debug/*.dll"
          ]
        }
      }
    }
  }

It doesn't seem to work with wildcards if not in a root directory

[BladeWise]

@MistyKuu You are right, the fun part is that this works

 "packOptions": {
    "files": {
      "mappings": {
        "mylib": {
          "include": [
            "bin/Debug*/*.dll"
          ]
        }
      }
    }
  }

I created a specific issue (#3184) regarding this.

[neethasp]

None of the foll. worked: - (1) "mappings": { "mylib": { "include": [ "bin/Debug/*.dll" ] } }

(2) { "mappings": { "dest/path": "./src/path" } }

(3) { "mappings": { "dest/path":{ "include":"./src/path" } } }

[rohit21agrawal]

@neethasp try using "\" instead of "/" and using includeFiles instead of include wherever you can.

[neethasp]

Dear Rohit,

Thank you.. I shall try this option.

Similarly how can the content of a nuget package be extracted into a local folder using the project.json? i.e., how can the foll. content of .csproj file be replaced with in project.json?: -

MyFolder\My.dll PreserveNewest

Regards, Neetha

On Thu, Jul 28, 2016 at 11:31 AM, Rohit Agrawal notifications@github.com wrote:

@neethasp https://github.com/neethasp try using "\" instead of "/" and using includeFiles instead of include wherever you can.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NuGet/Home/issues/3145#issuecomment-235806410, or mute the thread https://github.com/notifications/unsubscribe-auth/ALtEhH3tHpGMPy6AicLovagtmDh-yfhMks5qaEW7gaJpZM4JNmpF .

[rohit21agrawal]

@neethasp try something like this :

"packOptions": {
    "files": {
      "mappings": {
        "MyFolder": "..\\packages\\Mine\\My.dll"
      }
    }
  }

[neethasp]

Dear Rohit,

Please be informed that your suggestion of "'\' instead of '/' and using includeFiles instead of include wherever you can" did not work for adding files into nuget package using 'packOptions'.

Also your suggestion on extracting nuget content locally will not work because the 'packages' folder will no more be created with the migration to project.json.

Regards, Neetha S. Patill

On Wed, Aug 17, 2016 at 12:03 AM, Rohit Agrawal notifications@github.com wrote:

@neethasp https://github.com/neethasp try something like this :

"packOptions": { "files": { "mappings": { "MyFolder": "..\packages\Mine\My.dll" } } }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NuGet/Home/issues/3145#issuecomment-240195243, or mute the thread https://github.com/notifications/unsubscribe-auth/ALtEhAkkafutSJnKgm5T_3qgoui94jl7ks5qggJqgaJpZM4JNmpF .

[rohit21agrawal]

@neethasp could you please attach a repro project that highlights your issues? Would help a great deal to investigate the issue at hand. thanks!

[neethasp]

Dear Rohit,

Sorry but I shall not be able to provide the project related details. All I am trying to imply is it is not possible to pack/ unpack files in/ out of nuget packages as is possible with .nuspec file using the foll. elements: -

This is w.r.t retrieving the non dll files from nuget like 'My.txt' above and making use of them during project compilation.

Regards, Neetha S. Patil

On Fri, Aug 26, 2016 at 4:02 AM, Rohit Agrawal notifications@github.com wrote:

@neethasp https://github.com/neethasp could you please attach a repro project that highlights your issues? Would help a great deal to investigate the issue at hand. thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NuGet/Home/issues/3145#issuecomment-242564898, or mute the thread https://github.com/notifications/unsubscribe-auth/ALtEhKD2aysCqwJtBHGo-kcyX3Up2_5rks5qjhfzgaJpZM4JNmpF .

[rohit21agrawal]

ProjectA (1).zip

@neethasp it does work, please find an attached project which shows how the files tag works under packOptions

[prasannavl]

Considering that project.json is not the future, reverting back to the old behavior where .csproj is merged with nuspec should be ideal. I don't know why it was even removed in the first place. Its painful to keep changing nuget deployment systems.

This should be reopened, and fixed to merge with nuspec.

[prasannavl]

@rohit21agrawal

Note that we have fixed the nuget pack case for csproj + project.json, however, it is expected behavior that the nuspec file will be completely ignored.

It shouldn't be expected. I'd slap a bug label, if that's the expected behavior and reopen this issue. Considering nuspec is the sole isolated way of managing configuration without depending on the project systems, and also being the only command line way to be able to configure all metadata, it should be merged regardless of the project systems in play.