renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
17.44k stars 2.29k forks source link

Renovate adds issue links in release notes to GitHub, even when project hosted elsewhere #19503

Open aarongoldenthal opened 1 year ago

aarongoldenthal commented 1 year ago

How are you running Renovate?

Mend Renovate hosted app on github.com

If you're self-hosting Renovate, tell us what version of Renovate you run.

No response

If you're self-hosting Renovate, select which platform you are using.

None

If you're self-hosting Renovate, tell us what version of the platform you run.

No response

Was this something which used to work for you, and then stopped?

I never saw this working

Describe the bug

When release notes contain a reference to an issue without a specific link, e.g. Fixed something bad (#1), Renovate adds a link to the issue, but assumes the project is on GitHub, even if the release notes are hosted elsewhere (e.g. GitLab, and the notes were pulled from GitLab). As an example, this npm package is hosted on gitlab.com (as noted in the package).

Example project PR/MR illustrating the problem are listed below, which is slightly different using the renovate app on GitHub and self hosted on gitlab.com:

In both cases the issue links should point to https://gitlab.com/gitlab-ci-utils/gitlab-ci-env/, as noted in the package.

Relevant debug logs

Logs ``` DEBUG: No dangling containers to remove INFO: Repository started { "renovateVersion": "34.66.1" } DEBUG: Using localDir: /mnt/renovate/gh/aarongoldenthal/Renovate-GitHub-Issue-Link DEBUG: PackageFiles.clear() - Package files deleted DEBUG: initRepo("aarongoldenthal/Renovate-GitHub-Issue-Link") DEBUG: Using queue: host=api.github.com, concurrency=10 DEBUG: aarongoldenthal/Renovate-GitHub-Issue-Link default branch = main DEBUG: Using app token for git init DEBUG: Repository cache is restored from revision 13 DEBUG: Resetting npmrc DEBUG: checkOnboarding() DEBUG: isOnboarded() DEBUG: Checking cached config file name DEBUG: Existing config file confirmed DEBUG: Repository config { "fileName": "renovate.json", "config": { "$schema": "https://docs.renovatebot.com/renovate-schema.json", "extends": [ "config:base" ] } } DEBUG: Repo is onboarded DEBUG: migrateAndValidate() DEBUG: No config migration necessary DEBUG: massaged config { "config": { "$schema": "https://docs.renovatebot.com/renovate-schema.json", "extends": [ "github>whitesource/merge-confidence:beta", "config:base" ] } } DEBUG: migrated config { "config": { "$schema": "https://docs.renovatebot.com/renovate-schema.json", "extends": [ "github>whitesource/merge-confidence:beta", "config:base" ] } } DEBUG: Setting hostRules from config DEBUG: Found repo ignorePaths { "ignorePaths": [ "**/node_modules/**", "**/bower_components/**", "**/vendor/**", "**/examples/**", "**/__tests__/**", "**/test/**", "**/tests/**", "**/__fixtures__/**" ] } DEBUG: Using queue: host=api.github.com, concurrency=10 DEBUG: No vulnerability alerts found DEBUG: No vulnerability alerts found DEBUG: findIssue(Dependency Dashboard) DEBUG: Retrieving issueList DEBUG: Retrieved 2 issues DEBUG: Found issue 4 DEBUG: No baseBranches DEBUG: extract() DEBUG: Cached extract for sha=c2b730986d1db7327eb60ba14045d34aca7a8c1e is valid and can be used DEBUG: Deleted cached dep updates INFO: Dependency extraction complete { "baseBranch": "main", "stats": { "managers": { "npm": { "fileCount": 1, "depCount": 1 } }, "total": { "fileCount": 1, "depCount": 1 } } } DEBUG: PackageFiles.add() - Package file saved for base branch { "baseBranch": "main" } DEBUG: Package releases lookups complete { "baseBranch": "main" } DEBUG: branchifyUpgrades DEBUG: detectSemanticCommits() DEBUG: semanticCommits: returning "disabled" from cache DEBUG: 1 flattened updates found: gitlab-ci-env DEBUG: Returning 1 branch(es) DEBUG: config.repoIsOnboarded=true DEBUG: packageFiles with updates { "baseBranch": "main", "config": { "npm": [ { "constraints": { "npm": "<9" }, "deps": [ { "currentValue": "6.3.0", "currentVersion": "6.3.0", "datasource": "npm", "depIndex": 0, "depName": "gitlab-ci-env", "depType": "dependencies", "fixedVersion": "6.3.0", "isSingleVersion": true, "lockedVersion": "6.3.0", "prettyDepType": "dependency", "registryUrl": "https://registry.npmjs.org", "sourceUrl": "https://gitlab.com/gitlab-ci-utils/gitlab-ci-env", "versioning": "npm", "warnings": [], "updates": [ { "bucket": "non-major", "newVersion": "6.4.0", "newValue": "6.4.0", "releaseTimestamp": "2022-10-22T17:44:48.108Z", "newMajor": 6, "newMinor": 4, "updateType": "minor", "branchName": "renovate/gitlab-ci-env-6.x" } ] } ], "lockFiles": [ "package-lock.json" ], "managerData": { "hasPackageManager": false, "yarnZeroInstall": false }, "npmLock": "package-lock.json", "packageFile": "package.json", "packageFileVersion": "1.0.0", "packageJsonName": "renovate-github-issue-link", "skipInstalls": true } ] } } DEBUG: detectSemanticCommits() DEBUG: semanticCommits: returning "disabled" from cache DEBUG: processRepo() DEBUG: Processing 1 branch: renovate/gitlab-ci-env-6.x DEBUG: Calculating hourly PRs remaining DEBUG: getPrList success { "pullsTotal": 2, "requestsTotal": 1, "apiQuotaAffected": true } DEBUG: currentHourStart=2022-12-21T01:00:00.000+00:00 DEBUG: PR hourly limit remaining: 1 DEBUG: Calculating prConcurrentLimit (10) DEBUG: getBranchPr(renovate/gitlab-ci-env-6.x) DEBUG: findPr(renovate/gitlab-ci-env-6.x, undefined, open) DEBUG: Found PR #3 DEBUG: 1 PRs are currently open DEBUG: PR concurrent limit remaining: 9 DEBUG: Calculated maximum PRs remaining this run: 1 DEBUG: PullRequests limit = 1 DEBUG: Calculating hourly PRs remaining DEBUG: currentHourStart=2022-12-21T01:00:00.000+00:00 DEBUG: PR hourly limit remaining: 1 DEBUG: Calculating branchConcurrentLimit (10) DEBUG: 1 already existing branches found: renovate/gitlab-ci-env-6.x DEBUG: Branch concurrent limit remaining: 9 DEBUG: Calculated maximum branches remaining this run: 1 DEBUG: Branches limit = 1 DEBUG: syncBranchState()(branch="renovate/gitlab-ci-env-6.x") DEBUG: branch.isUpToDate(): using cached result "true"(branch="renovate/gitlab-ci-env-6.x") DEBUG: getBranchPr(renovate/gitlab-ci-env-6.x)(branch="renovate/gitlab-ci-env-6.x") DEBUG: findPr(renovate/gitlab-ci-env-6.x, undefined, open)(branch="renovate/gitlab-ci-env-6.x") DEBUG: Found PR #3(branch="renovate/gitlab-ci-env-6.x") DEBUG: branchExists=true(branch="renovate/gitlab-ci-env-6.x") DEBUG: dependencyDashboardCheck=undefined(branch="renovate/gitlab-ci-env-6.x") DEBUG: PR rebase requested=false(branch="renovate/gitlab-ci-env-6.x") DEBUG: Checking if PR has been edited(branch="renovate/gitlab-ci-env-6.x") DEBUG: branch.isModified(): using cached result "false"(branch="renovate/gitlab-ci-env-6.x") DEBUG: Found existing branch PR(branch="renovate/gitlab-ci-env-6.x") DEBUG: Checking schedule(at any time, null)(branch="renovate/gitlab-ci-env-6.x") DEBUG: No schedule defined(branch="renovate/gitlab-ci-env-6.x") DEBUG: Branch already exists(branch="renovate/gitlab-ci-env-6.x") DEBUG: getBranchPr(renovate/gitlab-ci-env-6.x)(branch="renovate/gitlab-ci-env-6.x") DEBUG: findPr(renovate/gitlab-ci-env-6.x, undefined, open)(branch="renovate/gitlab-ci-env-6.x") DEBUG: Found PR #3(branch="renovate/gitlab-ci-env-6.x") DEBUG: GET https://api.github.com/repos/aarongoldenthal/Renovate-GitHub-Issue-Link/branches/main/protection = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=150)(branch="renovate/gitlab-ci-env-6.x") DEBUG: No branch protection found(branch="renovate/gitlab-ci-env-6.x") DEBUG: Skipping behind base branch check due to rebaseWhen=auto(branch="renovate/gitlab-ci-env-6.x") DEBUG: isBranchConflicted(main, renovate/gitlab-ci-env-6.x)(branch="renovate/gitlab-ci-env-6.x") DEBUG: branch.isConflicted(): using cached result "false"(branch="renovate/gitlab-ci-env-6.x") DEBUG: Branch does not need rebasing(branch="renovate/gitlab-ci-env-6.x") DEBUG: Using reuseExistingBranch: true(branch="renovate/gitlab-ci-env-6.x") DEBUG: Checking if we can automerge branch(branch="renovate/gitlab-ci-env-6.x") DEBUG: mergeStatus=no automerge(branch="renovate/gitlab-ci-env-6.x") DEBUG: Ensuring PR(branch="renovate/gitlab-ci-env-6.x") DEBUG: There are 0 errors and 0 warnings(branch="renovate/gitlab-ci-env-6.x") DEBUG: getBranchPr(renovate/gitlab-ci-env-6.x)(branch="renovate/gitlab-ci-env-6.x") DEBUG: findPr(renovate/gitlab-ci-env-6.x, undefined, open)(branch="renovate/gitlab-ci-env-6.x") DEBUG: Found PR #3(branch="renovate/gitlab-ci-env-6.x") DEBUG: Found existing PR(branch="renovate/gitlab-ci-env-6.x") DEBUG: Fetching changelog: https://gitlab.com/gitlab-ci-utils/gitlab-ci-env (6.3.0 -> 6.4.0)(branch="renovate/gitlab-ci-env-6.x") DEBUG: Processing existing PR(branch="renovate/gitlab-ci-env-6.x") DEBUG: Pull Request #3 does not need updating(branch="renovate/gitlab-ci-env-6.x") DEBUG: PR is not configured for automerge(branch="renovate/gitlab-ci-env-6.x") DEBUG: getBranchPr(renovate/gitlab-ci-env-6.x) DEBUG: findPr(renovate/gitlab-ci-env-6.x, undefined, open) DEBUG: Found PR #3 DEBUG: branch.isBehindBase(): using cached result "false" DEBUG: isBranchConflicted(main, renovate/gitlab-ci-env-6.x) DEBUG: branch.isConflicted(): using cached result "false" DEBUG: Ensuring Dependency Dashboard DEBUG: ensureIssue(Dependency Dashboard) DEBUG: Issue is open and up to date - nothing to do DEBUG: Removing any stale branches DEBUG: config.repoIsOnboarded=true DEBUG: Branch lists { "branchList": [ "renovate/gitlab-ci-env-6.x" ], "renovateBranches": [ "renovate/gitlab-ci-env-6.x" ] } DEBUG: remainingBranches= DEBUG: No branches to clean up DEBUG: PackageFiles.clear() - Package files deleted DEBUG: Branch summary { "cacheModified": true, "baseBranches": [ { "branchName": "main", "sha": "c2b730986d1db7327eb60ba14045d34aca7a8c1e" } ], "branches": [ { "automerge": false, "baseBranch": "main", "baseBranchSha": "c2b730986d1db7327eb60ba14045d34aca7a8c1e", "branchName": "renovate/gitlab-ci-env-6.x", "branchSha": "8436aeebaa7f0d68a00d0419eb6c46eb8d6d00a3", "isModified": false, "isPristine": true } ], "inactiveBranches": [] } DEBUG: Renovate repository PR statistics { "stats": { "total": 1, "open": 1, "closed": 0, "merged": 0 } } DEBUG: Repository result: done, status: onboarded, enabled: true, onboarded: true DEBUG: Repository timing splits (milliseconds) { "splits": { "init": 3281, "extract": 1096, "lookup": 175, "onboarding": 1, "update": 503 }, "total": 5296 } DEBUG: Package cache statistics { "get": { "count": 3, "avgMs": 1, "medianMs": 1, "maxMs": 2 }, "set": { "count": 0 } } DEBUG: http statistics { "urls": { "https://api.github.com/graphql (POST,200)": 2, "https://api.github.com/repos/aarongoldenthal/Renovate-GitHub-Issue-Link/branches/main/protection (GET,404)": 1, "https://api.github.com/repos/aarongoldenthal/Renovate-GitHub-Issue-Link/contents/renovate.json (GET,200)": 1, "https://api.github.com/repos/aarongoldenthal/Renovate-GitHub-Issue-Link/issues/4 (GET,200)": 2, "https://api.github.com/repos/aarongoldenthal/Renovate-GitHub-Issue-Link/pulls (GET,200)": 1, "https://api.github.com/repos/whitesource/merge-confidence/contents/beta.json (GET,200)": 1 }, "hostStats": { "api.github.com": { "requestCount": 8, "requestAvgMs": 187, "queueAvgMs": 0 } }, "totalRequests": 8 } DEBUG: dns cache { "hosts": [ "api.github.com" ] } INFO: Repository finished { "cloned": false, "durationMs": 5296 }```

Have you created a minimal reproduction repository?

I have linked to a minimal reproduction repository in the bug description

github-actions[bot] commented 1 year ago

Hi there,

Get your issue fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction.

We may close the issue if you, or someone else, haven't created a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

aarongoldenthal commented 1 year ago

In case it wasn't clear above, there's a minimal reproduction using the GitHub app at https://github.com/aarongoldenthal/Renovate-GitHub-Issue-Link, and the same self-hosted using the GitLab templates at https://gitlab.com/aarongoldenthal/renovate-github-issue-link.

These both have a single NPM package and the baseline renovate config. Looking through other projects, it's not limited to NPM, it happens for other dependency types where GitLab release data is pulled for the release notes (it definitely also happens for GitLab includes).

rarkins commented 1 year ago

Thanks, reproduction forked to https://github.com/renovate-reproductions/19503

rarkins commented 1 year ago

This will need some refactoring of how we fetch and massage changelogs.

First of all, we massage the entire PR body here: https://github.com/renovatebot/renovate/blob/10ad86c44121a591d8d30c300d6df8388c6244ee/lib/workers/repository/update/pr/body/index.ts#L100

We could potentially just massage changelogs instead, or perhaps changelogs and pr body notes, but we'd be taking a risk of missing something.

Changelogs are generated earlier in the file here: https://github.com/renovatebot/renovate/blob/10ad86c44121a591d8d30c300d6df8388c6244ee/lib/workers/repository/update/pr/body/index.ts#L86

Note that there can be multiple upgrades within a branch, and they may be from different hosts.

So I can't see how to fix this without a pretty major refactor of release notes compilation

aarongoldenthal commented 1 year ago

If this issue is going to potentially cover a bigger refactor of release notes, one other related bug I saw in reviewing the links above is at https://github.com/renovatebot/renovate/blob/10ad86c44121a591d8d30c300d6df8388c6244ee/lib/modules/platform/gitlab/index.ts#L699

In the minimal reproduction a new release of the dependency came out that references CI_PROJECT_NAMESPACE_ID in the release notes, which is being changed in GitLab to CI_MROJECT_NAMESPACE_ID, so I believe this line should be something like this instead (I wasn't sure all of the cases it was trying to catch, the second was added to catch the PRs example in the current test):

    .replace(regEx(/\bPR\b/g), 'MR')
    .replace(regEx(/\bPRs\b/g), 'MRs')