search

dependabot / dependabot-core

🤖 Dependabot's core logic for creating update PRs.
https://docs.github.com/en/code-security/dependabot
MIT License
4.6k stars 972 forks source link

Include dependency version(s) in Git commit YAML #8217

Open martincostello opened 10 months ago

martincostello commented 10 months ago

Is there an existing issue for this?

Feature description

I have some automation where I auto-approve/merge/deploy changes from dependabot based on what is being updated. As part of this I need to know what the version of a dependency is.

For example, when a single dependency is updated for a NuGet package I get a commit message like this:

Bump Microsoft.Playwright from 1.38.0 to 1.39.0
Bumps [Microsoft.Playwright](https://github.com/microsoft/playwright-dotnet) from 1.38.0 to 1.39.0.
- [Release notes](https://github.com/microsoft/playwright-dotnet/releases)
- [Commits](https://github.com/microsoft/playwright-dotnet/compare/v1.38.0...v1.39.0)

---
updated-dependencies:
- dependency-name: Microsoft.Playwright
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

From the YAML I'm able to detect what is updated, except for the version, but I'm able to fetch the version from other parts of the commit message.

However, when dependency groups are used, this breaks down as the version information is no longer included anywhere in the commit message.

Bump the xunit group with 1 update
Bumps the xunit group with 1 update: [xunit](https://github.com/xunit/xunit).

- [Commits](https://github.com/xunit/xunit/commits)

---
updated-dependencies:
- dependency-name: xunit
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: xunit
...

Signed-off-by: dependabot[bot] <support@github.com>

To make such automation more robust and consistent, I propose that the version of the dependency that dependabot updates to is included in the YAML block for each dependency that is updated.

For example:

 Bump the xunit group with 1 update
 Bumps the xunit group with 1 update: [xunit](https://github.com/xunit/xunit).

 - [Commits](https://github.com/xunit/xunit/commits)

 ---
 updated-dependencies:
 - dependency-name: xunit
   dependency-type: direct:production
+  dependency-version: 2.5.3
   update-type: version-update:semver-patch
   dependency-group: xunit
 ...

 Signed-off-by: dependabot[bot] <support@github.com>
martincostello commented 10 months ago

If I could get some feedback on this issue whether it's a 👍 or a 👎, I'd be happy to push up a PR to implement it.

martincostello commented 8 months ago

I cannot for the life of me find where in the code that part of the commit message is written out so that I can try to amend it 😅 - any pointers as to where I should look?

JamieMagee commented 5 months ago

It looks like this is all configured before the Dependabot execution is queued as part of the commit message options. Here's the part of the code:

https://github.com/dependabot/dependabot-core/blob/95e8e792b12ac04577ed104fadd4951e57827dc1/common/lib/dependabot/pull_request_creator/message_builder.rb#L309-L316

martincostello commented 5 months ago

Thanks! I'll try and trace this back to find where the version information needs to go.

martincostello commented 5 months ago

Hmm - I don't quite follow how this is the place if it's coming from the dependabot configuration file - the version would be a property of the dependency resolved by running dependabot and finding something to update, wouldn't it?

JamieMagee commented 4 months ago

Ah, I thought it was configured as part of the CommitMessageOptions that was passed into the updater, but now I'm not sure.

martincostello commented 1 month ago

Been reminded of the fact I still haven't been able to find the source of this 😅

martincostello commented 1 month ago

So using the previous comment pointing to dependabot/cli as a clue, and running a job locally I think I understand a few more things I didn't before.

The CLI runs the updater, which does the updates and generates a PR description and commit message, but that ultimately just ends up in a JSON payload that is submitted to the dependabot API back-end which tells it what to change and that makes the real commits and API calls on the update process' behalf.

Running the following command locally:

go run cmd/dependabot/dependabot.go update nuget App-vNext/Polly -o out.yml

eventually gets me some output I can see in the terminal for some JSON like this (which I've prettified):

{
    "data": {
        "base-commit-sha": "fcf94b2a520166904256c7a7cea2c713959db7fc",
        "dependencies": [
            {
                "name": "xunit.runner.visualstudio",
                "previous-requirements": [
                    {
                        "file": "/eng/Test.targets",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.5.7",
                        "source": null
                    },
                    {
                        "file": "/test/Polly.Core.Tests/Polly.Core.Tests.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.5.7",
                        "source": null
                    },
                    {
                        "file": "/test/Polly.Extensions.Tests/Polly.Extensions.Tests.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.5.7",
                        "source": null
                    },
                    {
                        "file": "/test/Polly.RateLimiting.Tests/Polly.RateLimiting.Tests.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.5.7",
                        "source": null
                    },
                    {
                        "file": "/test/Polly.Specs/Polly.Specs.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.5.7",
                        "source": null
                    },
                    {
                        "file": "/test/Polly.Testing.Tests/Polly.Testing.Tests.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.5.7",
                        "source": null
                    },
                    {
                        "file": "/Directory.Packages.props",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.5.7",
                        "source": null
                    }
                ],
                "previous-version": "2.5.7",
                "requirements": [
                    {
                        "file": "/eng/Test.targets",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.8.2",
                        "source": {
                            "nuspec_url": "https://api.nuget.org/v3-flatcontainer/xunit.runner.visualstudio/2.8.2/xunit.runner.visualstudio.nuspec",
                            "source_url": null,
                            "type": "nuget_repo",
                            "url": "https://api.nuget.org/v3/index.json"
                        }
                    },
                    {
                        "file": "/test/Polly.Core.Tests/Polly.Core.Tests.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.8.2",
                        "source": {
                            "nuspec_url": "https://api.nuget.org/v3-flatcontainer/xunit.runner.visualstudio/2.8.2/xunit.runner.visualstudio.nuspec",
                            "source_url": null,
                            "type": "nuget_repo",
                            "url": "https://api.nuget.org/v3/index.json"
                        }
                    },
                    {
                        "file": "/test/Polly.Extensions.Tests/Polly.Extensions.Tests.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.8.2",
                        "source": {
                            "nuspec_url": "https://api.nuget.org/v3-flatcontainer/xunit.runner.visualstudio/2.8.2/xunit.runner.visualstudio.nuspec",
                            "source_url": null,
                            "type": "nuget_repo",
                            "url": "https://api.nuget.org/v3/index.json"
                        }
                    },
                    {
                        "file": "/test/Polly.RateLimiting.Tests/Polly.RateLimiting.Tests.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.8.2",
                        "source": {
                            "nuspec_url": "https://api.nuget.org/v3-flatcontainer/xunit.runner.visualstudio/2.8.2/xunit.runner.visualstudio.nuspec",
                            "source_url": null,
                            "type": "nuget_repo",
                            "url": "https://api.nuget.org/v3/index.json"
                        }
                    },
                    {
                        "file": "/test/Polly.Specs/Polly.Specs.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.8.2",
                        "source": {
                            "nuspec_url": "https://api.nuget.org/v3-flatcontainer/xunit.runner.visualstudio/2.8.2/xunit.runner.visualstudio.nuspec",
                            "source_url": null,
                            "type": "nuget_repo",
                            "url": "https://api.nuget.org/v3/index.json"
                        }
                    },
                    {
                        "file": "/test/Polly.Testing.Tests/Polly.Testing.Tests.csproj",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.8.2",
                        "source": {
                            "nuspec_url": "https://api.nuget.org/v3-flatcontainer/xunit.runner.visualstudio/2.8.2/xunit.runner.visualstudio.nuspec",
                            "source_url": null,
                            "type": "nuget_repo",
                            "url": "https://api.nuget.org/v3/index.json"
                        }
                    },
                    {
                        "file": "/Directory.Packages.props",
                        "groups": [
                            "dependencies"
                        ],
                        "requirement": "2.8.2",
                        "source": {
                            "nuspec_url": "https://api.nuget.org/v3-flatcontainer/xunit.runner.visualstudio/2.8.2/xunit.runner.visualstudio.nuspec",
                            "source_url": null,
                            "type": "nuget_repo",
                            "url": "https://api.nuget.org/v3/index.json"
                        }
                    }
                ],
                "version": "2.8.2"
            }
        ],
        "updated-dependency-files": [
            {
                "content": "\u003cProject\u003e\n  \u003cPropertyGroup\u003e\n    \u003cManagePackageVersionsCentrally\u003etrue\u003c/ManagePackageVersionsCentrally\u003e\n    \u003cPollyVersion\u003e8.4.1\u003c/PollyVersion\u003e\n    \u003cMicrosoftExtensionsVersion\u003e8.0.0\u003c/MicrosoftExtensionsVersion\u003e\n    \u003cMicrosoftExtensionsTimeProviderVersion\u003e8.7.0\u003c/MicrosoftExtensionsTimeProviderVersion\u003e\n  \u003c/PropertyGroup\u003e\n  \u003cItemGroup\u003e\n    \u003cPackageVersion Include=\"coverlet.msbuild\" Version=\"6.0.2\" /\u003e\n    \u003cPackageVersion Include=\"BenchmarkDotNet\" Version=\"0.13.12\" /\u003e\n    \u003cPackageVersion Include=\"FluentAssertions\" Version=\"6.12.0\" /\u003e\n    \u003cPackageVersion Include=\"FSharp.Core\" Version=\"8.0.200\" /\u003e\n    \u003cPackageVersion Include=\"GitHubActionsTestLogger\" Version=\"2.4.1\" /\u003e\n    \u003cPackageVersion Include=\"IcedTasks\" Version=\"0.11.4\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Bcl.AsyncInterfaces\" Version=\"6.0.0\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Bcl.TimeProvider\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.CodeAnalysis.BannedApiAnalyzers\" Version=\"3.3.4\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.CodeAnalysis.PublicApiAnalyzers\" Version=\"3.3.4\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.Caching.Memory\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.Configuration\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.DependencyInjection\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.Http.Resilience\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.Logging\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.Logging.Abstractions\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.Logging.Console\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.Options\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.Options.ConfigurationExtensions\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.Extensions.TimeProvider.Testing\" Version=\"$(MicrosoftExtensionsTimeProviderVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Microsoft.NET.Test.Sdk\" Version=\"17.10.0\" /\u003e\n    \u003cPackageVersion Include=\"MinVer\" Version=\"5.0.0\" /\u003e\n    \u003cPackageVersion Include=\"NSubstitute\" Version=\"5.1.0\" /\u003e\n    \u003cPackageVersion Include=\"Polly\" Version=\"$(PollyVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Polly.Core\" Version=\"$(PollyVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Polly.RateLimiting\" Version=\"$(PollyVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Polly.Extensions\" Version=\"$(PollyVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Polly.Testing\" Version=\"$(PollyVersion)\" /\u003e\n    \u003cPackageVersion Include=\"Polly.Contrib.WaitAndRetry\" Version=\"1.1.1\" /\u003e\n    \u003cPackageVersion Include=\"ReportGenerator\" Version=\"5.3.7\" /\u003e\n    \u003cPackageVersion Include=\"SonarAnalyzer.CSharp\" Version=\"9.28.0.94264\" /\u003e\n    \u003cPackageVersion Include=\"StyleCop.Analyzers\" Version=\"1.2.0-beta.556\" /\u003e\n    \u003cPackageVersion Include=\"System.ComponentModel.Annotations\" Version=\"4.5.0\" /\u003e\n    \u003cPackageVersion Include=\"System.Diagnostics.DiagnosticSource\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"System.Threading.RateLimiting\" Version=\"$(MicrosoftExtensionsVersion)\" /\u003e\n    \u003cPackageVersion Include=\"System.Threading.Tasks.Extensions\" Version=\"4.5.4\" /\u003e\n    \u003cPackageVersion Include=\"System.ValueTuple\" Version=\"4.5.0\" /\u003e\n    \u003cPackageVersion Include=\"xunit\" Version=\"2.7.0\" /\u003e\n    \u003cPackageVersion Include=\"xunit.runner.visualstudio\" Version=\"2.8.2\" /\u003e\n  \u003c/ItemGroup\u003e\n\u003c/Project\u003e\n",
                "content_encoding": "utf-8",
                "deleted": false,
                "directory": "/",
                "name": "Directory.Packages.props",
                "operation": "update",
                "support_file": false,
                "type": "file",
                "mode": ""
            }
        ],
        "pr-title": "Bump xunit.runner.visualstudio from 2.5.7 to 2.8.2",
        "pr-body": "Bumps xunit.runner.visualstudio from 2.5.7 to 2.8.2.\n",
        "commit-message": "Bump xunit.runner.visualstudio from 2.5.7 to 2.8.2\n\nBumps xunit.runner.visualstudio from 2.5.7 to 2.8.2.",
        "dependency-group": null
    },
    "type": "create_pull_request"
}

The interesting part (to me) is the commit message (formatted for readability):

Bump xunit.runner.visualstudio from 2.5.7 to 2.8.2

Bumps xunit.runner.visualstudio from 2.5.7 to 2.8.2.

There's no commit trailers or YAML here at all.

This suggests to me that the YAML and commit trailer are added by the process that is actually doing the GitHub API request to create the pull request.

I can't find any trace of that in the dependabot org, so I'm guessing that's a private repo here or in the github org that contains the backend server implementation and that is where the magic line of code to include the dependencies' version property in the metadata as dependency-version: $version needs to go.

If that's the case, then obviously I can't make that change myself as I'm not a GitHub employee 😄

@JamieMagee Does my analysis here sound correct, or am I barking up the wrong tree completely?

martincostello commented 4 weeks ago

@jakecoffman Sorry to ping you, but I was looking at #10195 in the release notes for the latest version and it seems like maybe that touches the code that might need the tweak for this feature request?

Am I correct? If so, any chance you could do that change to the service?