NuGet cannot find dotnet-watch, dotnet-user-secrets etc in NuGetFallbackFolder

[wli3]

Repo in preview 1 build - https://github.com/dotnet/versions/tree/master/build-info/dotnet/product/cli/release/2.1

Due to dotnet-watch, dotnet-user-secrets etc are not extracted with "restored format" in NuGetFallbackFolder

Actual

Result of dotnet install

C:\Users\wul>dotnet install tool -g dotnet-watch --version 2.1.0-preview1-28181
Install failed. Failed to download package:
NuGet returned:

Failed to restore package.
WorkingDirectory:
Arguments: restore C:\Users\wul\AppData\Local\Temp\zhdjmutp.geo\qgjwzesl.cop.csproj --runtime win10-x64 /p:BaseIntermediateOutputPath=\"C:\Users\wul\.dotnet\toolspkgs\dotnet-watch\2.1.0-preview1-28181\"
Output:   Restoring packages for C:\Users\wul\AppData\Local\Temp\zhdjmutp.geo\qgjwzesl.cop.csproj...
C:\Users\wul\AppData\Local\Temp\zhdjmutp.geo\qgjwzesl.cop.csproj : error NU1101: Unable to find package dotnet-watch. No packages exist with this id in source(s): <C:\Program Files\dotnet\sdk\NuGetFallbackFolder>
  Restore failed in 144.45 ms for C:\Users\wul\AppData\Local\Temp\zhdjmutp.geo\qgjwzesl.cop.csproj. 

Expected

C:\Program Files\dotnet\sdk\NuGetFallbackFolder\dotnet-watch>tree /f
Folder PATH listing
Volume serial number is 148E-B0BF
C:.
└───2.1.0-preview1-28181
    │   dotnet-watch.2.1.0-preview1-28181.nupkg
    │   dotnet-watch.2.1.0-preview1-28181.nupkg.sha512
    │   dotnet-watch.nuspec
    │
    └───tools
        └───any
            └───any
                │   dotnet-watch.deps.json
                │   dotnet-watch.dll
                │   dotnet-watch.pdb
                │   dotnet-watch.runtimeconfig.json
                │   DotnetToolSettings.xml
                │
                └───assets
                        DotNetWatch.targets

[wli3]

@muratg @natemcmaster

[wli3]

cc: @livarcocc

[wli3]

@emgarten @nkolev92 do you know what is the layout for packages in fallbackfolder as feed? I tried to move the nupkg to the root of fallbackfolder, but end up "cannot find Microsoft.NETCore.Platforms" (although Platforms is there as "restored" layout). In order to make it discover-able by nuget I have to run a restore on this nupkg, and copied the result of restore output to fallbackfolder

[nkolev92]

NuGet supports 2 different layouts for local folders.

One is a flat list(V2), the other one is the style(V3) like in the global packages folder.

The fallback folder is a V3 style folder, because it's equivalent to the global packages folder.

So something like: Root\PackageName\PackageVersion\ nupkg

You can look at the DG spec, to make sure that NuGet is picking up the right folder:

NUGET_PERSIST_DG=true
NUGET_PERSIST_DG_PATH=c:\tmp\output.dg (optional)

When you run a restore, it will write a spec that tells you what NuGet will use to restore.

[natemcmaster]

So the fallback folder appears to be following the V3 layout. We have $Root/dotnet-watch/2.1.0-preview1-28181/dotnet-watch.2.1.0-preview1-28181.nupkg on disk. Are we missing something else? Do I also need the nuspec extracted next to the nupkg?

[natemcmaster]

Also, @wli3 when dotnet-install-tool generates the csproj file used to restore a tools package, what property are you generating that points to C:\Program Files\dotnet\sdk\NuGetFallbackFolder\? Is it RestoreSources, RestoreProjectAdditionalSources, etc?

[nkolev92]

@natemcmaster Not of the top of my head. Extraction only matters if it's used as a fallback folder.

And that's exactly what I'm wondering, how does the fallback folder get added to nuget. The dependency graph spec should help us understand that.

[wli3]

by this

<RestoreAdditionalProjectSources>C:\Program Files\dotnet\sdk\NuGetFallbackFolder</RestoreAdditionalProjectSources>

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <RestorePackagesPath>C:\Users\wul\.dotnet\toolspkgs\dotnet-watch\2.1.0-preview1-28181</RestorePackagesPath>
    <RestoreProjectStyle>DotnetToolReference</RestoreProjectStyle>
    <RestoreRootConfigDirectory>C:\Program Files\dotnet</RestoreRootConfigDirectory>
    <DisableImplicitFrameworkReferences>true</DisableImplicitFrameworkReferences>
    <RestoreFallbackFolders>clear</RestoreFallbackFolders>
    <RestoreAdditionalProjectSources>C:\Program Files\dotnet\sdk\NuGetFallbackFolder</RestoreAdditionalProjectSources>
    <RestoreAdditionalProjectFallbackFolders></RestoreAdditionalProjectFallbackFolders>
    <RestoreAdditionalProjectFallbackFoldersExcludes></RestoreAdditionalProjectFallbackFoldersExcludes>
    <DisableImplicitNuGetFallbackFolder>true</DisableImplicitNuGetFallbackFolder>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="dotnet-watch" Version="2.1.0-preview1-28181" />
  </ItemGroup>
</Project>

[nkolev92]

@wli3 Can you please try what I suggested above and paste the dg spec here?

[natemcmaster]

I found it. @nkolev92 not sure if this a known bug or not, but apparently for NuGet to treat the nested folder layout as a v3 feed, you also need the .nuspec and .nupkg.sha512 files extracted next to the nupkg.

[wli3]

[wli3]

That's the minimal i find... and ya, you are right @natemcmaster

[natemcmaster]

cc @muratg assigning this to 2.1.0-preview1. Seems like a vital scenario we fix for this release. I think making this change in the LZMA generation is trivial, so I can take it on.

[wli3]

@nkolev92 here is dg spec of only nupkg under packageid/packageversion

{
  "format": 1,
  "restore": {
    "C:\\Users\\wul\\AppData\\Local\\Temp\\ce5wusmn.hb4\\ohlbuegj.ksa.csproj": {}
  },
  "projects": {
    "C:\\Users\\wul\\AppData\\Local\\Temp\\ce5wusmn.hb4\\ohlbuegj.ksa.csproj": {
      "version": "1.0.0",
      "restore": {
        "projectUniqueName": "C:\\Users\\wul\\AppData\\Local\\Temp\\ce5wusmn.hb4\\ohlbuegj.ksa.csproj",
        "projectName": "ohlbuegj.ksa",
        "projectPath": "C:\\Users\\wul\\AppData\\Local\\Temp\\ce5wusmn.hb4\\ohlbuegj.ksa.csproj",
        "packagesPath": "C:\\Users\\wul\\.dotnet\\toolspkgs\\dotnet-watch\\2.1.0-preview1-28181",
        "outputPath": "C:\\Users\\wul\\.dotnet\\toolspkgs\\dotnet-watch\\2.1.0-preview1-28181",
        "projectStyle": "DotnetToolReference",
        "configFilePaths": [
          "C:\\Users\\wul\\AppData\\Roaming\\NuGet\\NuGet.Config"
        ],
        "originalTargetFrameworks": [
          "netcoreapp2.1"
        ],
        "sources": {
          "C:\\Program Files\\dotnet\\sdk\\NuGetFallbackFolder": {}
        },
        "frameworks": {
          "netcoreapp2.1": {
            "projectReferences": {}
          }
        },
        "warningProperties": {
          "warnAsError": [
            "NU1605"
          ]
        }
      },
      "frameworks": {
        "netcoreapp2.1": {
          "dependencies": {
            "dotnet-watch": {
              "target": "Package",
              "version": "[2.1.0-preview1-28181, )"
            }
          },
          "imports": [
            "net461"
          ],
          "assetTargetFallback": true,
          "warn": true
        }
      },
      "runtimes": {
        "win10-x64": {
          "#import": []
        }
      }
    }
  }
}

[nkolev92]

@natemcmaster Yeah that's it.

@wli3 That looks good.

I think you guys got to the bottom of this.

[muratg]

@natemcmaster agreed that this should go in preview1. Though could you start an internal thread with Damian and Eilon for ask-mode approval?

[natemcmaster]

@muratg got approval.

This should be resolved https://github.com/aspnet/Universe/pull/829. @wli3 this should be in the next prodcon build of dotnet/cli.

Leaving open until we verify this in a build of dotnet/cli

[wli3]

@natemcmaster Will do. There should be one build tomorrow? Or just 5 hours from now?

[natemcmaster]

As soon as tomorrow, but could be longer. Hard to say because builds are so unstable right now.

[wli3]

Checked several times today, no new build. I will keep checking

[natemcmaster]

Hopefully we'll have a new build today. We have a 2.1 build of aspnetcore. We just need the CLI build to pass with our new bits

[natemcmaster]

Fixed in CLI 2.1.300-preview1-008019.

https://github.com/dotnet/versions/tree/master/build-info/dotnet/product/cli/release/2.1#product-build-cli-build-20180202-02-1347550.