search

NuGet / Home

Repo for NuGet Client issues
Other
1.49k stars 250 forks source link

Add `dotnet nuget push --dry-run` #13475

Open Smaug123 opened 3 months ago

Smaug123 commented 3 months ago

NuGet Product(s) Involved

dotnet.exe

The Elevator Pitch

Is your feature request related to a problem? Please describe.

I recently discovered only after a push to main that my dotnet nuget push command line was incorrect (I specified --skip-duplicates, a typo for --skip-duplicate).

Describe the solution you'd like

A new option: --dry-run, which does everything short of actually pushing the package.

Additional Context and Details

Additional context

It's normal for tools to offer --dry-run; for example, git push --dry-run, cargo publish --dry-run, npm publish --dry-run.

When writing automated pipelines to run in privileged environments, it's extremely helpful to be able to run the same pipeline in an unprivileged environment; the main alternative is to set up an entire parallel infrastructure (e.g. a separate NuGet repository), which is dramatically more work for the user. Concretely, if I could run dotnet nuget push ${DRY_RUN:+--dry-run} then I could have exactly the same code running on branches as on main.

BinToss commented 3 months ago

Note to maintainers: this issue was originally opened at https://github.com/dotnet/sdk/issues/41284 as a feature request for dotnet CLI.

I've worked around the lack of this feature by pushing a dummy package with --skip-duplicate. The exit code will be 0 if everything worked, including if the package already exists.

Pushing a pre-made, lightweight "PackageID=DUMMY" package works in few scenarios and has drawbacks "DUMMY" project, packing it, running it through Kuinox.NupkgDeterministicator, hashing the deterministic-ated nupkg, and storing the SHA256 and the nupkg in my shared dev-deps project and its package. With the following properties, I was able to get the generated nupkg down to 1.9 KiB with a SHA256 of `65A383F09CFBE6928619C620057A17D01C5A37704C5FEF1C99C53BB6E1BB6BA2`: ```xml DUMMY true false ``` If the hash doesn't match, the nupkg is regenerated and re-hashed. If it still doesn't match, throw. > Note: at the time of writing, `nuget.exe` and `dotnet nuget` do not have built-in deterministic pack—each time you run the pack target, the resulting nupkg will have a different hash. You'll need `Kuinox.NupkgDeterministicator` to make the package deterministic. This is included in the following script block. A re-usable command line to create the nupkg (if it doesn't exist) and hash would be something like... ```pwsh function new-dummy { # if DUMMY directory exists (due to failed commands?), delete it recursively if (Test-Path ./Dummy) { rmdir ./DUMMY -r } mkdir DUMMY; dotnet new classlib --framework net8.0 --output ./DUMMY/ $csproj = cat ./DUMMY/DUMMY.csproj -Raw $newline = $csproj.Contains("`r`n") ? "`r`n" : "`n" # create empty Content file so we can pack without libs or deps '' | Out-File -NoNewline ./DUMMY/DUMMY if (0 -ne (Get-Item ./DUMMY/DUMMY).Length) { throw "DUMMY file's length is greater than 0" } # include it in the project $includeLines = $newline + ' ' + $newline + ' ' + $newline ' ' + $newline; $csproj.Replace('' + $newline, '' + $newline + $includeLines) | Out-File -NoNewline ./DUMMY/DUMMY.csproj # Note: in .NET SDK 8 and later, `pack` sets Configuration to Release by default. Previous SDKs default to 'Debug'. This is partly due to the inclusion of SourceLink in .NET 8 and later. dotnet pack DUMMY -o ./ --no-build --property:Description=DUMMY --property:SuppressDependenciesWhenPacking=true --property:IncludeBuildOutput=false # cleanup dummy directory recursively rmdir ./DUMMY -r } if (-not (Test-Path './DUMMY.1.0.0.nupkg')) { new-dummy } if (-not (gcm NupkgDeterministicator 2> $null)) { dotnet tool install -g Kuinox.NupkgDeterministicator } NupkgDeterministicator ./DUMMY.1.0.0.nupkg $hash = Get-FileHash -Algorithm SHA256 ./DUMMY.1.0.0.nupkg $expected = '65A383F09CFBE6928619C620057A17D01C5A37704C5FEF1C99C53BB6E1BB6BA2'; if ($hash.Hash.ToUpperInvariant() -ne $expected) { new-dummy NupkgDeterministicator ./DUMMY.1.0.0.nupkg $hash = Get-FileHash -Algorithm SHA256 ./DUMMY.1.0.0.nupkg if ($hash.Hash.ToUpperInvariant() -ne $expected) { throw "DUMMY.1.0.0.nupkg SHA256 did not match, so the package was regenerated, but the new file's hash still did not match.`n" + "Expected: $($expected)`n" + "Actual : $($hash.Hash.ToUpperInvariant())" } } ```

EDIT: You cannot push PackageId "DUMMY" to Nuget.org without Write access to https://www.nuget.org/packages/DUMMY/.

Suggested --dry-run functionality

Dry run functionality for nuget pushes should be identical to a normal push, aside from the fact that the package should not be published after it is green-lit.

Bandaid solution in the meantime

Without a --dry-run argument, your Release pipeline should...

  1. (everything before preparing Release assets and packages)
  2. Check if the "$(PackageId)@$(PackageVersion)` (per package to be published) is already published on every source they will be published to. If remote duplicates are discovered, error out of the Release CI/CD and do not publicize anything. Maybe automate the creation of a ticket or an Issue like Semantic Release sometimes does.
    • This requires an API key with READ access for each Source. It doesn't matter for NuGet.org, but you need a valid key to query public GitHub NuGet package repositories.
  3. Pack dummy packages and push them with --skip-duplicates. Aside from a "dummy" version, these should be identical to the packages you publish later.
    • For each project, dotnet pack $(MSBuildProjectFullPath) --configuration Release --p:Version=0.0.1-DUMMY --skip-duplicates -o ./$(YourDummyNupkgsDirectory)/$(PackageId) (include symbols if desired).
    • For each (Source, ApiKey, PackageId) tuple, dotnet nuget push --source $(Source) --api-key $(ApiKey) "$(YourDummyNupkgsDirectory)/$(PackageId)" to push the dummy packages with to each source, using the correct key for each push. This seems odd, but it accounts for when you push multiple packages to one source and each package requires a different key.
    • These perform the following checks and then some:
      • Does not error if the dummy version already exists.
      • Verifies the package is a valid nupkg/snupkg.
      • Verifies an API key grants PUSH access for a particular given PackageId on a given Source.
        1. Additional pre-publish checks and other publish preparation (e.g. zipping release assets for a GitHub Release)
        2. Publish the new Release (incl. publishing the actual packages)