renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
17.65k stars 2.32k forks source link

Regex Manager depName mismatch when depName contains the value being updated #24559

Open rarkins opened 1 year ago

rarkins commented 1 year ago

Discussed in https://github.com/renovatebot/renovate/discussions/24533

Originally posted by **jpredmond** September 19, 2023 ### How are you running Renovate? Self-hosted Renovate ### If you're self-hosting Renovate, tell us what version of Renovate you run. 36.14.0 ### If you're self-hosting Renovate, select which platform you are using. GitLab self-hosted ### Was this something which used to work for you, and then stopped? I am trying to get this working for the first time ### Describe the problem ## Background: Our GitLab CI configuration file contains a number of variables that specify container image URLs, so that those images can be referenced via the variables in the "image:" sections of various jobs that use the same image without duplication. The standard managers do not interpret those variables, so we are attempting to use a regexManager to update these images. **Given** a .gitlab-ci.yml file with the following variable declaration: ```yaml variables: CUSTOM_IMAGE_RHEL7_X86_64: registry.gitlab.mycompany.com:443/our_group/a_subgroup/our_project/some_folder/image_name_ending_with_rhel7-x86_64:6 ``` **And given** the following in our renovate.json: ```json { "regexManagers": [ { "description": [ "All GitLab CI variables starting with 'CUSTOM_IMAGE_' referring to ", "images served from the our_project project container registry" ], "fileMatch": ["^\\.gitlab-ci\\.yml$"], "matchStrings": [ "\\n\\s*CUSTOM_IMAGE_\\S+: (?registry.gitlab.mycompany.com:443)/(?our_group/a_subgroup/our_project/some_folder/[^:]*?):(?\\d+)\\s" ], "datasourceTemplate": "docker" } ] } ``` ## Expected behavior: We expect that the tag (":7") in the variable value should be changed to the latest available tag (i.e., ":8"). ## Current behavior: The update fails, with the following message in the debug log: ``` "manager":"regex","packageFile":".gitlab-ci.yml","currentDepName":"our_group/a_subgroup/our_project/some_folder/image_name_ending_with_rhel7-x86_64","newDepName":"our_group/a_subgroup/our_project/some_folder/image_name_ending_with_rhel8-x86_64","msg":"depName mismatch" ``` ## Discussion: So apparently some step in the process is applying the change from "7" to "8" to the depName instead of (or in addition to) applying it to the tag. This behavior seems similar if not identical to the one documented in [Issue 8061](https://github.com/renovatebot/renovate/issues/8061), but that issue is marked as fixed. I don't know if there could have been a regression that caused the issue to reappear, or if this is a different root cause with the same symptom. In some discussions I found regarding that issue I saw a suggestion to add an autoReplaceStringTemplate setting to the regexManager, so I tried adding this line: ```json "autoReplaceStringTemplate": "{{{depName}}}:{{{newValue}}}", ``` This changed the error somewhat but didn't solve it. The resulting "depName mismatch" error indicated that with this addition to the configuration, the depName was being changed to a different depName in the same file. ``` "manager":"regex","packageFile":".gitlab-ci.yml","currentDepName":"our_group/a_subgroup/our_project/some_folder/image_name_ending_with_rhel7-x86_64","newDepName":"our_group/a_subgroup/our_project/some_folder/image_name_ending_with_ubuntu_20_04-x86_64","msg":"depName mismatch" ``` In my [minimal reproduction repository](https://github.com/jpredmond/renovate-depName-mismatch-repro) I am demonstrating both the original problem and the unsuccessful workaround at the same time, using variables starting with "IMAGE_TEST_" and "WORKAROUND_TEST_" respectively. ### Relevant debug logs
Logs ``` DEBUG: depName mismatch (branch="renovate/jpredmond-type_1_image-2.x") { "manager": "regex" "packageFile": ".gitlab-ci.yml" "currentDepName": "jpredmond/type_1_image" "newDepName": "jpredmond/type_2_image" } WARN: Error updating branch: update failure (branch="renovate/jpredmond-type_1_image-2.x") ```
### Have you created a minimal reproduction repository? I have linked to a minimal reproduction in the description above
rarkins commented 1 year ago

Reproduction: https://github.com/renovate-reproductions/24533

childnode commented 9 months ago

I would expect this can be easily reproduced in https://github.com/fgreinacher/renovate/blob/main/lib/modules/manager/dockerfile/extract.spec.ts ?

no?

looks like to me that there are two cases: somewhere in https://github.com/renovatebot/renovate/blob/c7ab3ca33711ab6783c7a5435230a8761b709a99/lib/workers/repository/update/branch/index.ts#L708 https://github.com/renovatebot/renovate/blob/c7ab3ca33711ab6783c7a5435230a8761b709a99/lib/workers/repository/update/branch/auto-replace.ts#L182

  1. with customManager autoReplaceStringTemplate the new registry is queried, newContent but somewhere after like in https://github.com/renovatebot/renovate/blob/c7ab3ca33711ab6783c7a5435230a8761b709a99/lib/workers/repository/update/branch/auto-replace.ts#L340 the confirmIfDepUpdated reuses the old matcher and detects the next dependency not being processed yet. That perhaps causes the "depName mismatch"

in some constellations I see updater ends up in things like Could not extract {{packageFile}} (manager=regex) after autoreplace. Did the autoreplace make the file unparseable?

This might be caused by https://github.com/renovatebot/renovate/pull/17883

  1. related to this but if using the packageRule replacementNameTemplate no repository replacement is made if entitled in the match - perhaps thats caused by #14149 -> #24883 implicit too, because regex is not named to be supported yet!!