Versions Maven Plugin not showing latest plugins available with ignores.

[garretwilson]

On Windows 10 with Maven 3.9.1 I upgraded to org.codehaus.mojo:versions-maven-plugin:2.15.0.

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>versions-maven-plugin</artifactId>
  <version>2.15.0</version>
</plugin>

I wanted to see what plugins were out of date, so I ran:

mvn versions:display-plugin-updates

It shows this:

[INFO] The following plugin updates are available:
[INFO]   maven-failsafe-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   maven-release-plugin ............................ 2.5.3 -> 3.0.0-M6
[INFO]   maven-site-plugin .............................. 3.12.1 -> 4.0.0-M3
[INFO]   maven-surefire-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> 3.0.5

So I added ignores for milestone releases as discussed in #684 and originally in #258:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>versions-maven-plugin</artifactId>
  <version>2.15.0</version>
  <configuration>
    <ruleSet>
      <ignoreVersions>
        <ignoreVersion><type>regex</type><version>.*-SNAPSHOT</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-alpha(?:-?\d+)?</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-beta(?:-?\d+)?</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-M(?:-?\d+)?</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-RC(?:-?\d+)?</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-pfd(?:-?\d+)?</version></ignoreVersion>
      </ignoreVersions>
    </ruleSet>
  </configuration>
</plugin>

Now when I run mvn versions:display-plugin-updates it gives me:

[INFO] No plugins require a newer version of Maven than specified by the pom.

This is wrong on several counts. org.apache.maven.plugins:maven-failsafe-plugin for example has v3.0.0 and 3.1.0 available—two versions released after the current version.

And the bug apparently has nothing to do with ignoring the *-M* versions, because it even skipped Spring Boot Maven Plugin, which is currently at v3.1.0.

This would appear critically broken, in that it does not show new versions, which is the essential function of this goal.

Or did I get the regular expressions wrong somehow? (I think they can be simplified a bit; I was starting from those in a Stack Overflow answer. But in any case I don't see how these regular expressions would be hiding versions in the form 3.0.5.)

[garretwilson]

Thank you for all your work on this, @cstamas ! It's so good to know we're getting somewhere. I had put some work on hold to help get this resolved, as I didn't want to force everything to update until we knew the source of the problem.

@ajarmoniuk , thanks for your input and steering me in the right direction, and letting us hijack this ticket for a bit.

Created https://issues.apache.org/jira/browse/MRESOLVER-363 This is a resolver bug, nothing to do with versions plugin. So if you agree, please close this issue.

I'll go continue my testing of the new Versions Maven Plugin regex ignore version feature, and if that is working, I'll note that and close this ticket. Further discussion on this bug can happen at MRESOLVER-363.

Thanks again, and I hope you have a good, restful night @cstamas .

[andrzejj0]

Awesome work! Congrats on getting to the bottom of it!

[cstamas]

So in Maven 3.9.1 this is happening to everyone who already has metadata? Some particular update to Maven Resolver basically left current metadata "locked" at its present state? Am I understanding that correctly?

Is the reason that you could not reproduce this earlier the fact that you had no metadata at all, and it was updating everything from scratch? But the moment anyone has metadata, it gets "locked" going forward, is that right?

No, as on issue: reaolver did not found value for lastUpdated. This as we saw, may happen due several factors:

• Not present resolver-status.propertiee
• or is present, but created with maven-compat (uses different key)

So is not locked for "all, who already have the metadata". If lastUpdated found, it will work. The conditions above must be met.

Again, local repo state may be altered by many apps, not only maven resolver, so it is def not "insanely hard" to get into situation like this, but still, it is not "locked down to all who downloaded it once" either.

[garretwilson]

Just to confirm: the latest understanding is that pull request #957 will fix this problem, because it will override the default policy of local repository metadata to say that the metadata should be refreshed at least daily (including if no last-update information is found by Maven Artifact Resolver)?

(The follow-up question of course is whether there is an idea of when that change might be included in a new release?)

[slawekjaranowski]

Just to confirm: the latest understanding is that pull request #957 will fix this problem, because it will override the default policy of local repository metadata to say that the metadata should be refreshed at least daily (including if no last-update information is found by Maven Artifact Resolver)?

(The follow-up question of course is whether there is an idea of when that change might be included in a new release?)

I have plan a next release in this week 😄

[garretwilson]

I have plan a next release in this week 😄

Wonderful! But can you confirm that we expect #957 to fix this bug? I just wanted to make sure I understood the current status.

[slawekjaranowski]

Please look - https://github.com/mojohaus/versions/pull/965#issuecomment-1564372249 And my way to testing - https://github.com/mojohaus/versions/pull/965#issuecomment-1564400340

So I assume the #957 - will fix problem with missing or wrong property in resolver-status.properties

[garretwilson]

I saw that v2.16.0 was released yesterday, so here goes the test … 🥁

First here is mvn versions:display-plugin-updates with v2.15.0:

[INFO] --- versions:2.15.0:display-plugin-updates (default-cli) @ … ---
[INFO]
[INFO] The following plugin updates are available:
[INFO]   maven-failsafe-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   maven-release-plugin ............................ 2.5.3 -> 3.0.0-M6
[INFO]   maven-site-plugin .............................. 3.12.1 -> 4.0.0-M3
[INFO]   maven-surefire-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> 3.0.5

Now here is mvn versions:display-plugin-updates with v2.16.0. (I'm really trying it as I write this. Oh the suspense … 😅 )

[INFO] --- versions:2.16.0:display-plugin-updates (default-cli) @ … ---
…
[INFO]
[INFO] The following plugin updates are available:
[INFO]   com.akathist.maven.plugins.launch4j:launch4j-maven-plugin  2.1.3 -> 2.4.1
[INFO]   maven-assembly-plugin .............................. 3.4.2 -> 3.6.0
[INFO]   maven-compiler-plugin ............................ 3.10.1 -> 3.11.0
[INFO]   maven-dependency-plugin ............................ 3.3.0 -> 3.6.0
[INFO]   maven-deploy-plugin ................................ 3.0.0 -> 3.1.1
[INFO]   maven-enforcer-plugin .............................. 3.1.0 -> 3.3.0
[INFO]   maven-failsafe-plugin ............................. 2.22.2 -> 3.1.2
[INFO]   maven-gpg-plugin ................................... 3.0.1 -> 3.1.0
[INFO]   maven-install-plugin ............................... 3.0.1 -> 3.1.1
[INFO]   maven-jar-plugin ................................... 3.2.2 -> 3.3.0
[INFO]   maven-javadoc-plugin ............................... 3.4.1 -> 3.5.0
[INFO]   maven-release-plugin ............................... 2.5.3 -> 3.0.1
[INFO]   maven-resources-plugin ............................. 3.3.0 -> 3.3.1
[INFO]   maven-site-plugin .............................. 3.12.1 -> 4.0.0-M8
[INFO]   maven-source-plugin ................................ 3.2.1 -> 3.3.0
[INFO]   maven-surefire-plugin ............................. 2.22.2 -> 3.1.2
[INFO]   org.codehaus.mojo:build-helper-maven-plugin ........ 3.3.0 -> 3.4.0
[INFO]   org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> 3.1.0

Oh, wow! Apparently I was behind in a lot of plugins and never new it!! But it looks like #957 fixed the problem with the stale update from Maven Central. In particular we see com.akathist.maven.plugins.launch4j:launch4j-maven-plugin 2.1.3 -> 2.4.1, which is the plugin we were using as an example in this ticket. And very interesting: if we look in ~/.m2/repository/com/akathist/maven/plugins/launch4j/launch4j-maven-plugin/resolver-status.properties, here is what it looked like before:

#Last modified on: Thu Sep 08 15:34:35 PDT 2022
#Thu Sep 08 15:34:35 PDT 2022
central.maven-metadata-central.xml.lastUpdated=1662676475074

Here is what ~/.m2/repository/com/akathist/maven/plugins/launch4j/launch4j-maven-plugin/resolver-status.properties looks like now:

#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Wed Jun 07 13:39:37 PDT 2023
central.maven-metadata-central.xml.lastUpdated=1662676475074
maven-metadata-central.xml.lastUpdated=1686170377857

Basically there is a new maven-metadata-central.xml.lastUpdated, and the "compatibility layer" property central.maven-metadata-central.xml.lastUpdated (which was what it was ignoring before) has been left untouched. I'm sure that's fine.

During this run of mvn versions:display-plugin-updates, because of #957 it went and downloaded lots of metadata XML from Maven Central, but (whew!) not an inordinate number. Basically it seems to have just pulled down metadata for these plugins that were out of date. I'm assuming those were the plugins missing the maven-metadata-central.xml.lastUpdated as well.

OK, things seem to be working great so far. Now for the grand finale: adding ignores so that I don't see things like the milestone releases. You'll remember this was the original point of all this back in #258 and #684.

<configuration>
  <ruleSet>
    <ignoreVersions>
      <ignoreVersion><type>regex</type><version>.*-SNAPSHOT</version></ignoreVersion>
      <ignoreVersion><type>regex</type><version>.*-alpha(?:-?\d+)?</version></ignoreVersion>
      <ignoreVersion><type>regex</type><version>.*-beta(?:-?\d+)?</version></ignoreVersion>
      <ignoreVersion><type>regex</type><version>.*-M(?:-?\d+)?</version></ignoreVersion>
      <ignoreVersion><type>regex</type><version>.*-RC(?:-?\d+)?</version></ignoreVersion>
      <ignoreVersion><type>regex</type><version>.*-pfd(?:-?\d+)?</version></ignoreVersion>
    </ignoreVersions>
  </ruleSet>
</configuration>

And running mvn versions:display-plugin-updates with those ignores:

[INFO] --- versions:2.16.0:display-plugin-updates (default-cli) @ … ---
[INFO]
[INFO] The following plugin updates are available:
[INFO]   com.akathist.maven.plugins.launch4j:launch4j-maven-plugin  2.1.3 -> 2.4.1
[INFO]   maven-assembly-plugin .............................. 3.4.2 -> 3.6.0
[INFO]   maven-compiler-plugin ............................ 3.10.1 -> 3.11.0
[INFO]   maven-dependency-plugin ............................ 3.3.0 -> 3.6.0
[INFO]   maven-deploy-plugin ................................ 3.0.0 -> 3.1.1
[INFO]   maven-enforcer-plugin .............................. 3.1.0 -> 3.3.0
[INFO]   maven-failsafe-plugin ............................. 2.22.2 -> 3.1.2
[INFO]   maven-gpg-plugin ................................... 3.0.1 -> 3.1.0
[INFO]   maven-install-plugin ............................... 3.0.1 -> 3.1.1
[INFO]   maven-jar-plugin ................................... 3.2.2 -> 3.3.0
[INFO]   maven-javadoc-plugin ............................... 3.4.1 -> 3.5.0
[INFO]   maven-release-plugin ............................... 2.5.3 -> 3.0.1
[INFO]   maven-resources-plugin ............................. 3.3.0 -> 3.3.1
[INFO]   maven-source-plugin ................................ 3.2.1 -> 3.3.0
[INFO]   maven-surefire-plugin ............................. 2.22.2 -> 3.1.2
[INFO]   org.codehaus.mojo:build-helper-maven-plugin ........ 3.3.0 -> 3.4.0
[INFO]   org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> 3.1.0

You'll notice that maven-site-plugin is no longer listed. Unfortunately I can't confirm 100% whether it's working correctly, because if you ignore 4.0.0-M8 then I already have the latest version 3.12.1 specified, so we don't know if it would show a greater version than this, e.g. 3.12.2. But from @ajarmoniuk 's test earlier in this ticket, it did appear to do what it was supposed to do.

OK this is really, great news, especially since this has been a journey of over five years (starting with #258). Thank you everyone involved! I'm finally going to be able to close my own ticket, JAVA-241: Configure Versions Maven Plugin to ignore non-release versions.. 🎉