Unable to use GitHub Packages V1 for binary cache

[donny-dont]

[!IMPORTANT]
This issue is specific to GitHub Packages V1 which is used in private GitHub Enterprise instances. Public GitHub uses V2 of GitHub Packages and does not have this issue.

I'm attempting to use GitHub packages as a binary cache following the tutorial. The NuGet feed is on a local GitHub Enterprise installation.

From there I run.

vcpkg install my-port --triplet x64-windows --debug

The results with identifying data taken out look like

[DEBUG] 1015: CreateProcessW("C:\GitHub\tools\nuget\nuget.exe" push -ForceEnglishOutput -Verbosity detailed -NonInteractive "C:\Dev\vcpkg\buildtrees\my-port_x64-windows.1.1.0-vcpkgb410183cd6ad6634474c5ba7f5d54e02ffff69d29086eec9d3e4a26bbea698ff.nupkg" -Timeout 100 -Source https://HOSTNAME/_registry/nuget/NAMESPACE/index.json/index.json)
NuGet Version: 6.10.0.107
Pushing my-port_x64-windows.1.1.0-vcpkgb410183cd6ad6634474c5ba7f5d54e02ffff69d29086eec9d3e4a26bbea698ff.nupkg to 'https://HOSTNAME/_registry/nuget/NAMESPACE/index.json'...
  PUT https://HOSTNAME/_registry/nuget/NAMESPACE/index.json/
WARNING: No destination repository detected. Ensure the source project has a 'RepositoryUrl' property defined. If you're using a nuspec file, ensure that it has a repository element with the required 'type' and 'url' attributes.
  BadRequest https://HOSTNAME/_registry/nuget/NAMESPACE/index.json/ 506ms
Response status code does not indicate success: 400 (Bad Request).
System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at NuGet.Protocol.Core.Types.PackageUpdateResource.EnsureSuccessStatusCode(HttpResponseMessage response, Nullable`1 codeNotToThrow, ILogger logger)
   at NuGet.Protocol.Core.Types.PackageUpdateResource.<>c__DisplayClass26_0.<PushPackageToServer>b__0(HttpResponseMessage response)
   at NuGet.Protocol.HttpSource.<ProcessResponseAsync>d__20`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.Core.Types.PackageUpdateResource.<PushPackageToServer>d__26.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Protocol.Core.Types.PackageUpdateResource.<PushPackageCore>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at NuGet.Protocol.Core.Types.PackageUpdateResource.<PushPackagePath>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at NuGet.Protocol.Core.Types.PackageUpdateResource.<Push>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()https://HOSTNAME/_registry/nuget/NAMESPACE/index.json
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.Commands.PushRunner.<Run>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.CommandLine.PushCommand.<ExecuteCommandAsync>d__36.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NuGet.CommandLine.Command.Execute()
   at NuGet.CommandLine.Program.MainCore(String workingDirectory, String[] args)
[DEBUG] 1015: cmd_execute_and_stream_data() returned 1 after  1325765 us
Pushing NuGet to "https://HOSTNAME/_registry/nuget/NAMESPACE/index.json/index.json" failed. Use --debug for more information.

The API logs report WARNING: No destination repository detected which implied that within the NAMESPACE there needed to be a repository named my-port_x64-windows. The documentation doesn't refer to any naming requirements for the repository so this was unknown to me.

Trying again the command still failed with the same error as ☝️ .

I had attempted to use an Artifactory feed as a binary cache earlier, which I could push to but it would never restore from since it was unable to find the package, so I downloaded that package. I unzipped it and added to the generated .nuspec a RepositoryUrl to tell GitHub packages where things were meant to go.

<repository type="git" url="https://HOSTNAME/NAMESPACE/my-port_x64-windows" />

I created a package from that with the additional feed and was able to manually push it to GitHub packages and saw the result within the GitHub enterprise instance's UI.

So I think there are two issues happening here. One is on the GitHub side where it doesn't seem to be able to figure out where things should go based on the <port>_<triplet>.<version>-vcpkg<hash> which is being discussed with our support contact. One is on the vcpkg side where it doesn't seem to be able to specify a cache repo, or if there is a way it isn't documented.

Thanks in advance for any help!

Related discussion 👇

Discussed in https://github.com/microsoft/vcpkg/discussions/23285

^{Originally posted by **equeim** February 26, 2022} I use vcpkg with NuGet binary caching via GitHub Packages in my GitHub actions workflow. It works fine with single repository but fails with multiple repos due to repository association of uploaded NuGet packages. I.e. repository Foo runs Actions workflow in which vcpkg builds some packages and automatically uploads them to GitHub Packages using NuGet. Now these packages are associated with repository Foo. Now if repository Bar has the same vcpkg dependencies with same versions, then when its workflow is run vcpkg can successfully download them from GitHub Packages. If I then update versions of these dependencies in repository Bar, then vcpkg will rebuild them and fail to upload new versions to GitHub Packages because they are already associated with repository Foo. How can I circumvent this issue?

[donny-dont]

@JonLiu1993 since you applied a repro label I tried to make a reproduction case and tried using public GitHub to do a NuGet feed. The public GitHub seems to allow a package that isn't associated with a repository while the GitHub Enterprise offering does not.

Internally we have a support ticket on this with GitHub so we can prolly close this.

[JonLiu1993]

The reason I added this label is that I successfully exported the package locally using the command ./vcpkg export zlib:x64-windows --nuget, and I created a C++ console project and used it normally, but I don’t have the environment to upload it to GitHub Packages to reproduce your issue.

PS F:\Feature-test\vcpkg> ./vcpkg export zlib:x64-windows --nuget
The following packages are already built and will be exported:
  * vcpkg-cmake:x64-windows
    zlib:x64-windows
Additional packages (*) need to be exported to complete this operation.
Exporting vcpkg-cmake:x64-windows...
Exporting zlib:x64-windows...
Creating NuGet package...
Created nupkg: F:\Feature-test\vcpkg\vcpkg-export-20240605-023504.1.0.0.nupkg

PM> Install-Package "vcpkg-export-20240605-023504" -Source "F:\Feature-test\vcpkg"

Attempting to gather dependency information for package 'vcpkg-export-20240605-023504.1.0.0' with respect to project 'nuget-project', targeting 'native,Version=v0.0'
Gathering dependency information took 6 ms
Attempting to resolve dependencies for package 'vcpkg-export-20240605-023504.1.0.0' with DependencyBehavior 'Lowest'
Resolving dependency information took 0 ms
Resolving actions to install package 'vcpkg-export-20240605-023504.1.0.0'
Resolved actions to install package 'vcpkg-export-20240605-023504.1.0.0'
Retrieving package 'vcpkg-export-20240605-023504 1.0.0' from 'F:\Feature-test\vcpkg'.
Adding package 'vcpkg-export-20240605-023504.1.0.0' to folder 'C:\Users\test\source\repos\nuget-project\packages'
Added package 'vcpkg-export-20240605-023504.1.0.0' to folder 'C:\Users\test\source\repos\nuget-project\packages'
Added package 'vcpkg-export-20240605-023504.1.0.0' to 'packages.config'
Successfully installed 'vcpkg-export-20240605-023504 1.0.0' to nuget-project
Executing nuget actions took 854 ms
Time Elapsed: 00:00:00.9872465

[donny-dont]

The reason I added this label is that I successfully exported the package locally using the command ./vcpkg export zlib:x64-windows --nuget, and I created a C++ console project and used it normally, but I don’t have the environment to upload it to GitHub Packages to reproduce your issue.

GitHub Packages is available to all. If you go through the setup described in the docs you should be off and running in case you have to debug any issues in the future.

From speaking to the GitHub Support Team it appears that github.com is using v2 of GitHub Packages while the GitHub Enterprise offering is using v1. In v1 a package needs to be associated with a repository while in v2 that association isn't required. That's why I'm seeing an error when pushing to a GitHub Enterprise instance but not to github.com.

[donny-dont]

@JonLiu1993 the response from the GitHub engineers was that having <repository> in the .nuspec was necessary so I'll prolly start hacking on the vcpkg-tool repo. Updating the title and description so it refers to V1 of GitHub Packages since V2 works perfectly fine.

[JonLiu1993]

Thanks for your inquiry, please ping me if you need any help.

[github-actions[bot]]

This is an automated message. Per our repo policy, stale issues get closed if there has been no activity in the past 28 days. The issue will be automatically closed in 14 days. If you wish to keep this issue open, please add a new comment.