Detect and follow Maven group and artifact moves

[ChristianMurphy]

What would you like Renovate to be able to do? Some maven artifacts change group and artifact ids on major releases. Many projects include a link forward to the new location, for example the "Note: This artifact was moved to" sections of:

• https://mvnrepository.com/artifact/commons-collections/commons-collections
• https://mvnrepository.com/artifact/org.apache.commons/commons-io
• https://mvnrepository.com/artifact/org.apache.commons/commons-math

Describe the solution you'd like When group and artifacts have their next major version linked in maven central, allow renovate updates to upgrade to the new version at the new group/artifact location.

Describe alternatives you've considered N/A

Additional context

NOTE: this applies to all maven based package managers, including gradle and sbt

[rarkins]

Is essentially the same concept as #5535 ?

[ChristianMurphy]

Similar concept, this would not be a manual rename, but could be automatically extracted from maven central. More related to https://github.com/renovatebot/renovate/issues/2223

[ChristianMurphy]

ran into this on https://github.com/Jasig/SimpleContentPortlet/pull/152

[rarkins]

OK, we would need to get that info out of the API somehow though.

[ChristianMurphy]

It looks like this can be detected through the relocation metadata https://maven.apache.org/guides/mini/guide-relocation.html

[rarkins]

OK, waiting on #2223

[froque]

There is a maven plugin, oga-maven-plugin, to detect these artifact moves.

It works with a curated JSON file: https://github.com/jonathanlermitage/oga-maven-plugin/blob/master/uc/og-definitions.json

[nielsbasjes]

Yesterday (almost) all of my projects got the same update from renovate that broke the CI build.

Update dependency pl.project13.maven:git-commit-id-plugin to v4.9.9

all of them failed with

Error:  Plugin pl.project13.maven:git-commit-id-plugin:4.9.9 or one of its dependencies could not be resolved: Could not find artifact io.github.git-commit-id:git-commit-id-maven-plugin:jar:4.9.9 in central (https://repo.maven.apache.org/maven2) -> [Help 1]

The rootcause was that this plugin got moved to a different package. The people who build this software did this in exactly the way Maven tells you:

https://maven.apache.org/guides/mini/guide-relocation.html

They created a 4.9.9 version in the old package which is ONLY a pom.xml with a redirect and does not contain any jar/war/... files. The redirect in this case looks like this:

<distributionManagement>
  <relocation>
    <groupId>io.github.git-commit-id</groupId>
    <artifactId>git-commit-id-maven-plugin</artifactId>
  </relocation>
</distributionManagement>

The effect can for example be seen here https://search.maven.org/artifact/pl.project13.maven/git-commit-id-plugin which shows:

Relocated to    io.github.git-commit-id:git-commit-id-maven-plugin

@rarkins since this is apparently the standard way of doing this I would really like this maven standard way to be supported by renovate.

[github-actions[bot]]

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[rarkins]

Can you create a minimal reproduction of that case? We are still blocked on a generic way to "replace" packages though

[nielsbasjes]

@rarkins This is the smallest I could think of (including renovate and a failing CI build): https://github.com/nielsbasjes/Renovate5667/ https://github.com/nielsbasjes/Renovate5667/pull/3

The full error here is

[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------< nl.basjes.bugreports:renovate5667 >------------------
[INFO] Building renovate5667 0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/pl/project13/maven/git-commit-id-plugin/4.9.9/git-commit-id-plugin-4.9.9.pom
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/pl/project13/maven/git-commit-id-plugin/4.9.9/git-commit-id-plugin-4.9.9.pom (1.4 kB at 1.3 kB/s)
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/pl/project13/maven/git-commit-id-plugin-parent/4.9.9/git-commit-id-plugin-parent-4.9.9.pom
[INFO] Downloaded from central: https://repo.maven.apache.org/maven2/pl/project13/maven/git-commit-id-plugin-parent/4.9.9/git-commit-id-plugin-parent-4.9.9.pom (2.5 kB at 189 kB/s)
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/io/github/git-commit-id/git-commit-id-maven-plugin/4.9.9/git-commit-id-maven-plugin-4.9.9.pom
Warning:  The POM for io.github.git-commit-id:git-commit-id-maven-plugin:jar:4.9.9 is missing, no dependency information available
[INFO] Downloading from central: https://repo.maven.apache.org/maven2/io/github/git-commit-id/git-commit-id-maven-plugin/4.9.9/git-commit-id-maven-plugin-4.9.9.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.388 s
[INFO] Finished at: 2021-07-12T15:41:42Z
[INFO] ------------------------------------------------------------------------
Error:  Plugin pl.project13.maven:git-commit-id-plugin:4.9.9 or one of its dependencies could not be resolved: Could not find artifact io.github.git-commit-id:git-commit-id-maven-plugin:jar:4.9.9 in central (https://repo.maven.apache.org/maven2) -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
Error: Process completed with exit code 1.

[github-actions[bot]]

Thank you for providing a reproduction! :tada: :rocket:

The Renovate team will take a look at the reproduction repository.

[TheSnoozer]

Hello, I'm one of the remaining maintainers of the git-commit-id-plugin project. Let me start with that the maven relocation guide really needs improvement. Whatever I have done or messed up, please do not treat it as "standard". I really have no idea what I'm doing this was my first "official" and perhaps last relocation.

IMHO I just wanted to make users aware that the plugin moved to new coordinates and thus published a relocation pom only. I intentionally did not publish a jar since I wanted to keep the option open to publish hotfix releases with old coordinates (I really can't be asked to change the coordinates for old releases). As a result my perhaps stupid idea was to publish a "4.9.9"-release containing the relocation info only. Under the new coordinates only a "5.0.0" exists. Perhaps one could resolve the "can't find artifact"-issue if I would publish a 4.9.9 under the new coordinates.

If this works (now confirmed) then the relocation can be one of the following publication-types: 1) A (full) publication with a jar and a normal pom stating the relocation (see below). 2) A pom-only publication (no dependencies declared, essentially a naked pom as I have had it published) pointing to the new location, however the jar is residing under the new location

Edit: I can confirm that after publishing the plugin to the new coordinates (https://github.com/git-commit-id/git-commit-id-maven-plugin/issues/570) one can use the old artifact/group combination or new. So in the case of 2 the relocation acts as "symlink" or whatever you want to call it.

I don't know what would happen if one published the jar to the old coordinates and the new coordinates....

Note: The relocation itself can be boiled down to the following snippet:

<distributionManagement>
  <relocation>
    <groupId>io.github.git-commit-id</groupId>
    <artifactId>git-commit-id-maven-plugin</artifactId>
  </relocation>
</distributionManagement>

For the full pom refer to https://repo1.maven.org/maven2/pl/project13/maven/git-commit-id-plugin/4.9.9/git-commit-id-plugin-4.9.9.pom

[astellingwerf]

Now that #5558 is merged into the trunk, I'd like to take a stab at this one. I had already worked on this this summer, but @JamieMagee was not fully done with the replacement impl back then.

[astellingwerf]

Watchers, can you guide me a bit, as I'm not too familiar with the standard development practices in this community?

There are the datasource aspects of this task, and then there are the managers. The datasource was easy to get done, but now that it detects the replacement, none (but the NPM changes set up by @JamieMagee) of the managers know how to handle it. The DD will propose to create a PR, but the manager is gonna ignore the replacement name while generating the proposed new code.

What's the default approach for phasing in such changes? Would it make sense to add a boolean property supportsReplacements to the ManagerApi interface that declares the support for replacing dependencies?

[rarkins]

Yes, could be good to add that flag and log a warning if we find an attempted replacement for a manager which doesn't support it.

[JamieMagee]

I'm not sure we require a new flag. I was looking into this for Docker, and this is the key codepath:

https://github.com/renovatebot/renovate/blob/ab018191fac6961b5f9d0a73261ac8fba0b4566f/lib/workers/branch/get-updated.ts#L97-L103

If a manager has a custom updateDependency method defined, we use that, otherwise use doAutoReplace. In the case of npm and maven managers, they do have a custom updateDependency. That's where you'll need to check if upgrade.newName is defined and handle it.

https://github.com/renovatebot/renovate/blob/aa56d49a1607cb4adf2cbad98180c140e80c6a12/lib/manager/maven/update.ts#L27-L42

That should be the only change required for this issue.

[rarkins]

FWIW (not related to Maven) I'm assuming that autoReplace could be enhanced to support a replacement name in a similar way it today can support a new value/version. I would suggest we do them as two separate routines, i.e. don't try to replace both package name and version at once.