Maven: Update versions defined in a property but managed by parent pom

sephiroth-j commented 2 years ago

What would you like Renovate to be able to do?

For example, a typical use case would be Spring Boot.

You have defined a project like Spring Boot as a parent project that defines, for example, the foo.version property that is used in a managed dependency.

parent POM

    <properties>
        <foo.version>1.2.3</foo.version>
    </properties>

    <dependencyManagement>
        <dependencies>
...
            <dependency>
                <groupId>foo.group</groupId>
                <artifactId>foo-artifact</artifactId>
                <version>${foo.version}</version>
            </dependency>
...
        </dependencies>
    </dependencyManagement>

In the child POM file, this dependency is included as usual, and since it is managed, the version element is omitted. But you also overwrite the foo.version property, since the value in the parent POM does not refer to the latest version at this time.

child POM

    <parent>
        <groupId>parent.group</groupId>
        <artifactId>parent-artifact</artifactId>
        <version>1.0.0</version>
        <relativePath />
    </parent>
    <properties>
        <foo.version>1.2.4</foo.version>
    </properties>

    <dependencies>
...
        <dependency>
            <groupId>foo.group</groupId>
            <artifactId>foo-artifact</artifactId>
        </dependency>
...
    </dependencies>

Later, Renovate updates the version of the parent file to 1.0.1. This updated parent POM now contains a newer version defined in foo.version (e.g. 1.2.5). Unfortunately, this update is skipped because the child POM still uses the old version it defined before.

Parent projects of the same repository, means which can be resolved using their relative path are out of scope! Because Renovate would already perform the update in the parent itself.

It would be great if Renovate could also detect this kind of dependencies and update the version defined in the child pom. This should only happen if the property is defined/overridden in the child pom - if it is not there, nothing should be done. This avoids updating all other dependencies that might be defined and outdated in the parent pom.

If you have any ideas on how this should be implemented, please tell us here.

check if there is a dependency without a version element in the current pom
check if this dependency is managed in a parent outside of the current repository (empty relativePath) and if it uses a property for specifying the version
check if this property is used in the child where we started
if so, check if there is an update available for this dependency and update the property in the child. this should be done the same way as if the dependency would have been defined directly with the version element.

It may not be sufficient to check only the direct ancestor, but it may be necessary to check its ancestor as well. There should be a limit to how many ancestors Renovate should check to solve this kind of managed dependency to avoid long or even endless runs.

Is this a feature you are interested in implementing yourself?

No

github-actions[bot] commented 2 years ago

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

rarkins commented 2 years ago

Where do these child/parent pom's from your description live?

sephiroth-j commented 2 years ago

The child pom lives in my own git repository which is processed by Renovate. The parent, and this is the crucial part, lives outside my sphere of influence and that of Renovate. The parent is a foreign Maven dependency.

This POM here would be an example of such a child. In this example I would like to see that Renovate detects that there is a newer version 21.5.0.0 of com.oracle.database.jdbc:ojdbc11 and updates the property oracle-database.version to "21.5.0.0".

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.6.7</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>spring-boot-complete</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>spring-boot-complete</name>
    <description>Demo project for Spring Boot</description>

    <properties>
        <java.version>11</java.version>
        <oracle-database.version>21.4.0.0.1</oracle-database.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>com.oracle.database.jdbc</groupId>
            <artifactId>ojdbc11</artifactId>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

rarkins commented 2 years ago

Thanks. Please put this into a minimal repo which we can fork and debug later (instructions are above).

I see that there's <oracle-database.version>21.4.0.0.1</oracle-database.version> in your pom.xml but no further reference to it within the file. What exact logic should Renovate use to know that oracle-database.version means com.oracle.database.jdbc:ojdbc11? e.g. I looked into what I think is the parent pom https://repo1.maven.org/maven2/org/springframework/boot/spring-boot-starter-parent/2.6.7/spring-boot-starter-parent-2.6.7.pom but see no reference for oracle-database.version. Please excuse in case I've missed something, as I'm not a Maven user. This is why we request full reproduction repos.

sephiroth-j commented 2 years ago

I created a sample project at https://github.com/InversoGmbH/renovate-issue-15170-demo

The spring-boot-starter-parent has itself spring-boot-dependencies as parent which defines all those properties and manages the dependencies - either directly or by importing another POM such as com.oracle.database.jdbc:ojdbc-bom.

As described in the initial comment, Renovate would need to check the parents of the current POM for a managed dependency that uses the property in question. If the direct parent does not contain such a dependency, the grand parent needs to be checked etc. That's why it is good idea to add an (configurable) limit for the numbers of parents to check.

rarkins commented 2 years ago

Thanks. The good news is that I think I understand it now, the bad news is that this will be very challenging for anybody to implement.

What we know at extraction time:

There's a parent package
There's a foo.version with no references in the current pom

Meanwhile somewhere up the parent tree, that foo.version may be used for one or more packages.

Today that variable would be ignored in our list of extracted dependencies because no dependency references it - it wouldn't show up as a dependency. As a general rule, we don't want to do any lookups from networks during our extract phase, so that means it would need to be done somehow during the lookup phase.

Maybe one day someone can work out an elegant way to implement this, but in the meantime I think you'll need to use regexManagers for this.

rarkins commented 2 years ago

Reproduction forked to https://github.com/renovate-reproductions/15170

sephiroth-j commented 2 years ago

Okay. I already thought that it will not be so easy. I will have a look at regexManagers, thanks for the tip.

lfvjimisola commented 2 years ago

If you are ok to handle this explicitly then use the regExp matcher below. I haven't tested that maven datasource uses depName=groupId and packageName=artifactId.

    {
        "fileMatch": [
            "(^|/|\\.)pom\\.xml$", "^(((\\.mvn)|(\\.m2))/)?settings\\.xml$"
        ],
        "matchStrings": [
            ".*<!-- renovate: datasource=(?<datasource>.*?) groupId=(?<depName>.*?) artifactId=(?<packageName>.*?) -->.*[\\r\\n]{1,2}.*<.*\\.version>(?<currentValue>.*?)<\/.*\\.version>"
        ]
    }

    <properties>
        <!-- renovate: datasource=maven groupId=org.apache.camel.springboot artifactId=camel-spring-boot-starter -->
        <camel.version>3.18.2</camel.version>
    </properties>

sephiroth-j commented 2 years ago

Thanks for the suggestion but ...

datasource is not needed and can be set to a fixed value, same for versioningTemplate.
comments are just comments, and anything on a separate line or before or after the actual property element is error prone - e.g. when merging.
an XML processing instruction could be used instead of a comment because it has semantic, but still has the same problem as no. 2.
using an attribute with its own namespace directly on the property element keeps everything together and is a clean, robust solution.

So here is my "solution" for Maven that works very well.

pom.xml

<project>
...
    <properties xmlns:renovate="https://docs.renovatebot.com/renovate-schema">
        <foo.version renovate:depName="foo.group:foo-artifcat">1.2.3</foo.version>
    </properties>
...
</project>

renovate.json

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "regexManagers": [{
    "fileMatch": ["(^|/|\\.)pom\\.xml$"],
    "matchStrings": ["<\\S+\\.version renovate:depName=\"(?<depName>\\S+?)\">(?<currentValue>.+?)<\\/\\S+\\.version>"],
    "versioningTemplate": "maven",
    "datasourceTemplate": "maven"
  }]
}

p.s. A note about the regex: The example is intentionally kept simple. Backtracking, for example, is not supported by the RE2 regex engine used by Renovate.

renovatebot / renovate