search

mojohaus / versions

Versions Maven Plugin
https://www.mojohaus.org/versions/versions-maven-plugin/
Apache License 2.0
337 stars 267 forks source link

Versions Maven Plugin not showing latest plugins available with ignores. #959

Closed garretwilson closed 1 year ago

garretwilson commented 1 year ago

On Windows 10 with Maven 3.9.1 I upgraded to org.codehaus.mojo:versions-maven-plugin:2.15.0.

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>versions-maven-plugin</artifactId>
  <version>2.15.0</version>
</plugin>

I wanted to see what plugins were out of date, so I ran:

mvn versions:display-plugin-updates

It shows this:

[INFO] The following plugin updates are available:
[INFO]   maven-failsafe-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   maven-release-plugin ............................ 2.5.3 -> 3.0.0-M6
[INFO]   maven-site-plugin .............................. 3.12.1 -> 4.0.0-M3
[INFO]   maven-surefire-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> 3.0.5

So I added ignores for milestone releases as discussed in #684 and originally in #258:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>versions-maven-plugin</artifactId>
  <version>2.15.0</version>
  <configuration>
    <ruleSet>
      <ignoreVersions>
        <ignoreVersion><type>regex</type><version>.*-SNAPSHOT</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-alpha(?:-?\d+)?</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-beta(?:-?\d+)?</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-M(?:-?\d+)?</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-RC(?:-?\d+)?</version></ignoreVersion>
        <ignoreVersion><type>regex</type><version>.*-pfd(?:-?\d+)?</version></ignoreVersion>
      </ignoreVersions>
    </ruleSet>
  </configuration>
</plugin>

Now when I run mvn versions:display-plugin-updates it gives me:

[INFO] No plugins require a newer version of Maven than specified by the pom.

This is wrong on several counts. org.apache.maven.plugins:maven-failsafe-plugin for example has v3.0.0 and 3.1.0 available—two versions released after the current version.

And the bug apparently has nothing to do with ignoring the *-M* versions, because it even skipped Spring Boot Maven Plugin, which is currently at v3.1.0.

This would appear critically broken, in that it does not show new versions, which is the essential function of this goal.

Or did I get the regular expressions wrong somehow? (I think they can be simplified a bit; I was starting from those in a Stack Overflow answer. But in any case I don't see how these regular expressions would be hiding versions in the form 3.0.5.)

andrzejj0 commented 1 year ago

Should we maybe use compile scope for Maven dependencies in this plugin?

cstamas commented 1 year ago

Am more and mote convinced that versions plugin is "good as is", but there is something else that has foul play (extension?)

cstamas commented 1 year ago

@garretwilson if no other idea, please reproduce whole DEBUG log (from beginning to end) where you had these lines:

[DEBUG] Checking com.akathist.maven.plugins.launch4j:launch4j-maven-plugin for updates newer than 2.1.3
[DEBUG] Determining update check for artifact com.akathist.maven.plugins.launch4j:launch4j-maven-plugin (C:\Users\user\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin\maven-metadata-central.xml) from central (https://repo.maven.apache.org/maven2)
[DEBUG] Searching for central.maven-metadata-central.xml.lastUpdated in resolution tracking file.
[DEBUG] Reading resolution-state from: C:\Users\user\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin\resolver-status.properties
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].version=2.1.3

And send it to me or anyone on this issue even in private (you have our emails from Maven ML) or if nothing special in contents, just attach it as gist.

garretwilson commented 1 year ago

[DEBUG] Found ignored versions: .*-SNAPSHOT (regex), .*-alpha(?:-?\d+)? (regex), .*-beta(?:-?\d+)? (regex), .*-M(?:-?\d+)? (regex), .*-RC(?:-?\d+)? (regex), .*-pfd(?:-?\d+)? (regex)

Just a note to try to prevent possible confusion on the outputs. There are couple of POMs being used here. Initially I opened this ticket using a POM with org.codehaus.mojo:versions-maven-plugin:2.15.0 using its new "ignore versions" feature. The output above probably is using that POM.

Once I realized that there was a bigger issue at play, in order to reduce the variables I switched to using an older POM, which I reported on the mailing list as well, which using an older version v2.12.0 Versions Maven Plugin and does not have any version includes.

Let me switch to org.codehaus.mojo:versions-maven-plugin:2.15.0 in the older POM and run it again:

[DEBUG] Calculated minimum version 3.5.0 from version parameter value '3.5.0'
[DEBUG] Checking com.akathist.maven.plugins.launch4j:launch4j-maven-plugin for updates newer than 2.1.3
[DEBUG] Could not find metadata com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml in local (C:\Users\user\.m2\repository)
[DEBUG] Skipped remote request for com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml, locally cached metadata up-to-date
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].version=2.1.3
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].artifactVersion=2.1.2
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].effectiveVersion=2.1.3
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].specified=true

So perhaps the difference in the output is that I had moved back to the old POM using org.codehaus.mojo:versions-maven-plugin:2.12.0?

In any case updating to org.codehaus.mojo:versions-maven-plugin:2.15.0 produces output without the "Determining update check …" and without the "Reading resolution-state …", but the problem still persists. So perhaps those lines were coming from org.codehaus.mojo:versions-maven-plugin:2.12.0, but apparently they have no bearing on the problem as I still get:

[INFO] The following plugin updates are available:
[INFO]   maven-failsafe-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   maven-release-plugin ............................ 2.5.3 -> 3.0.0-M6
[INFO]   maven-site-plugin .............................. 3.12.1 -> 4.0.0-M3
[INFO]   maven-surefire-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> 3.0.5

No mention of com.akathist.maven.plugins.launch4j:launch4j-maven-plugin being out of date.

cstamas commented 1 year ago

We both worked against POM you pointed us at: https://bitbucket.org/globalmentor/globalmentor-root/src/main/pom.xml

So I guess debug output of this one would be needed IF it produces same debug output as in quoted bits.

garretwilson commented 1 year ago

@cstamas:

your environment

Windows 10 Professional 64-bit WSL 2 installed but not being used Docker Desktop installed but not running Eclipse EE 2023-03 installed but not running for these tests

exact java, maven version (3.9.1 I guess, like it was mentioned somewhere)

Apache Maven 3.9.1 (2e178502fcdbffc201671fb2537d0cb4b4cc58f8)
Maven home: C:\…\maven
Java version: 17.0.4, vendor: Eclipse Adoptium, runtime: C:\…\jdk-17
Default locale: en_US, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

maven settings.xml

<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <server>
      <id>ossrh</id>
      <username>…</username>
      <password>…</password>
    </server>
  </servers>

  <profiles>
    <!-- (profile commented out)  -->
  </profiles>
</settings>

possible maven extensions installed (maven and build, and all, in /lib or .mvn or whatever)

Absolutely none whatsoever. This is a pure Maven unzip using the given POM.

whatever else that comes to your mind 😄

Believe me I'm going through everything I can think of. Please see the enumeration of files and timestamps on the list and let me know if anything there gives you an idea.

cstamas commented 1 year ago

And one more telling difference (and proof legacy code works for @garretwilson): from: https://github.com/mojohaus/versions/issues/959#issuecomment-1561928213

He's resolver-status.properties file:

#Last modified on: Thu Sep 08 15:34:35 PDT 2022
#Thu Sep 08 15:34:35 PDT 2022
central.maven-metadata-central.xml.lastUpdated=1662676475074

Observe:

In turn, my resolver-status.properties is:

#NOTE: This is a Maven Resolver internal implementation file, its format can be changed without prior notice.
#Thu May 25 16:27:01 CEST 2023
maven-metadata-px-oss.xml.lastUpdated=1685024821632

Observer:

So log messages and the "touch" properties file proves @garretwilson uses legacy code. Question remains: why?

garretwilson commented 1 year ago

We both worked against POM you pointed us at: https://bitbucket.org/globalmentor/globalmentor-root/src/main/pom.xml

That version had the new Versions Maven Plugin ignores, which could definitely be confusing (I could imagine people responding "ah, there's a bug in the Versions Maven Plugin that is incorrectly ignoring things" like I thought at first), so on the list I indicated https://github.com/globalmentor/globalmentor-root/blob/bce5bdbac7797b5b9114a72e5da2f4d76f3e24a7/pom.xml, which is exactly the same except it doesn't have the version ignores—but also uses an older version of Maven Versions Plugin, which apparently gives different debug output as you indicated. So let me go through this:

cstamas commented 1 year ago

Ok, that is to be expected, as 2.14 moved to maven resolver (from legacy) https://github.com/mojohaus/versions/releases/tag/2.14.0

garretwilson commented 1 year ago

So log messages and the "touch" properties file proves @garretwilson uses legacy code. Question remains: why?

I want to emphasize that I was using org.codehaus.mojo:versions-maven-plugin:2.12.0 until now. It was time to update this POM to the latest versions, so only now did I upgrade to org.codehaus.mojo:versions-maven-plugin:2.15.0 and file this ticket thinking that v2.15.0 had a bug in its new ignore-version functionality.

Isn't one simple explanation that org.codehaus.mojo:versions-maven-plugin:2.12.0 used legacy code (hence the different log output and recent updates to the local cache timestamps), but that org.codehaus.mojo:versions-maven-plugin:2.15.0 uses non-legacy code (hence the different log output)?

Nevertheless both org.codehaus.mojo:versions-maven-plugin:2.12.0 and org.codehaus.mojo:versions-maven-plugin:2.15.0 incorrectly show the wrong versions on my machine.

garretwilson commented 1 year ago

@cstamas , when you get a chance (not trying to rush you), take one last glance over the files I sent on the mailing list and let me know if any of them look out of place.

Do we have any idea which file org.codehaus.mojo:versions-maven-plugin:2.15.0 should be looking at in its stack to determine if Maven Central needs to be contacted?

Then if you have no further ideas I'm going to start mucking with the files in the ~/.m2/repository directory, one at a time, to see which lever I pull makes Versions Maven Plugin start seeing that com.akathist.maven.plugins.launch4j:launch4j-maven-plugin needs updating. I'm only making sure we cover all the possibilities before I start doing that, because once it gets "uncorrupted" I have no guarantee that we can go back and reproduce the "corrupted" state.

garretwilson commented 1 year ago

@cstamas on the mailing list (I think you accidentally sent the message directly to me and not to the list) you said:

And finally, the update (in resolver) is simple: It is the resolver-status.properties next to cached files key is $fileName + ".lastIpdated" value is unix epoch timestamp

But how can this be? In my C:\Users\user\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin directory tree I have no files ending in .lastUpdated. (The only thing close is m2e-lastUpdated.properties in the subdirectory for the individual version.)

In fact in my entire ~/.m2/repositories tree of over 25,000 files, almost all the files I have ending in .lastUpdated are my own libraries that I have published to Maven. There are a few others like Netty and Lucene, but just a tiny few.

Oh, but here is something super interesting. In ~/.m2/repository/org/slf4j/slf4j-api, along with the normal version subdirectories (e.g. ~/.m2/repository/org/slf4j/slf4j-api/2.0.4 and the maven-metadata-central.xml, maven-metadata-central.xml.sha1, and resolver-status.properties files, I have these subdirectories, with file system timestamps of 2022-09-19:

Wow, something screwy definitely going on here. Yes, those directories are actually named #{, $, and ${slf. It looks like there was some attempt at string interpolation gone wrong. Maybe I didn't supply a version value in a typo in one version of the POM? In any case, I don't know that this would have any affect on the update for com.akathist.maven.plugins.launch4j:launch4j-maven-plugin.

And since most of my ~/.m2/repository tree has no .lastUpdated files,are you sure you found the correct code path for this cache staleness checking?

garretwilson commented 1 year ago

I'm going to take an exercise break, and @cstamas you're probably ready to turn in for the night, so we can continue the discussion tomorrow. I just want to see if anyone thinks of anything else after resting, before I start mucking with the ~/.m2/repository tree.

cstamas commented 1 year ago

Filename is resolver-status.properties It is a java properties file. Key is...

cstamas commented 1 year ago

Ok, so to recap: debug logs are 2.12 output (that does use.legacy maven infra), and we tried (and failed) to reproduce this with 2.15 (as pom you pointed at had 2.15)

Still, question remains, why none did not discover newer versions. Will try tomorrow some combo of mixed (legacy and new) use over same local repository

garretwilson commented 1 year ago

Filename is resolver-status.properties

Ah, OK. So here goes … removing that file should definitely make the related com.akathist.maven.plugins.launch4j:launch4j-maven-plugin:2.1.3 see that it needs updating to v2.4.1, right? So 🥁 I'm renaming resolver-status.properties to resolver-status.properties.bad and issuing mvn versions:display-plugin-updates > debug-rename-resolver-status-properties.log --debug …

Nope. That still gives me:

…
[DEBUG] Checking com.akathist.maven.plugins.launch4j:launch4j-maven-plugin for updates newer than 2.1.3
[DEBUG] Could not find metadata com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml in local (C:\Users\user\.m2\repository)
[DEBUG] Skipped remote request for com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml, locally cached metadata up-to-date
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].version=2.1.3
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].artifactVersion=2.1.2
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].effectiveVersion=2.1.3
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].specified=true
…
[INFO] The following plugin updates are available:
[INFO]   maven-failsafe-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   maven-release-plugin ............................ 2.5.3 -> 3.0.0-M6
[INFO]   maven-site-plugin .............................. 3.12.1 -> 4.0.0-M3
[INFO]   maven-surefire-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> 3.0.5

It's still "[s]kipp[ing] remote request" for com.akathist.maven.plugins.launch4j:launch4j-maven-plugin. In addition it did not regenerate resolver-status.properties—it is still "missing" (i.e. renamed).

cstamas commented 1 year ago

No, rename it dir higher, in com/akathist/mavenlplugins/launch4j/launch4j-maven-plugin

garretwilson commented 1 year ago

No, rename it dir higher

I don't know if you mean to rename resolve-status.properties to a higher directory, or to rename a directory at a higher level. Sure, I'll get to renaming a higher directory level, but that's the equivalent of deleting the whole directory. If it's true that really everything that relates to whether the cache is stale is within com/akathist/mavenlplugins/launch4j/launch4j-maven-plugin, then eventually I should be able to change/delete a file that makes it go to the server, no? If I just blow away the directory, we don't know what force the update.

cstamas commented 1 year ago

Rename properties file dir level higher

garretwilson commented 1 year ago

Rename properties file dir level higher

There is no properties file a dir level higher for me to rename. Do you mean move the properties file up a level higher?

I'll try that in just a second. First an update …

cstamas commented 1 year ago

There is no properties file in com/akathist/maven/plugins/launch4j/launch4j-maven-plugin?

Can you list that dir?

garretwilson commented 1 year ago

There is no properties file in com/akathist/mavenlplugins/launch4j/launch4j-maven-plugin?

Sorry, maybe I wasn't clear earlier. The resolver-status.properties file and the maven-metadata-central.xml properties file etc. are in ~\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin. There are no files in ~\.m2\repository\com\akathist\maven\plugins\launch4j\.

cstamas commented 1 year ago

Ok, so pls rename properties file in ~.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin

Only the resolver-status.properties

garretwilson commented 1 year ago

Update: renaming maven-metadata-central.xml and maven-metadata-central.xml.sha1 ~did not fix the problem~ (oh wait, it did fix the problem! see below), but Maven did regenerate maven-metadata-central.xml, maven-metadata-central.xml.sha1, and resolver-status.properties! I find this interesting for several reasons:

garretwilson commented 1 year ago

Ok, so pls rename properties file in ~.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin

I did exactly that above. That was the first thing I did once I started mucking around with the ~/.m2 directory tree.

cstamas commented 1 year ago

So, maven does not regenerate XML from top of he's head, it was downloaded. Can you paste related log output (hopefully debug)

To rinse-repeat, just repeat steps you did before and add -X

garretwilson commented 1 year ago

Oh, wait wait wait. I'm getting ahead of myself. v2.4.1 is the latest. So it would appear renaming maven-metadata-central.xml above did the trick and forced Maven to contact Maven Central

I just checked the original maven-metadata.central.xml, and it says <release>2.1.2</release>. The new one says <latest>2.4.1</latest> So it is working (for this artifact). And the debug output says:

[DEBUG] Checking com.akathist.maven.plugins.launch4j:launch4j-maven-plugin for updates newer than 2.1.3
[DEBUG] Could not find metadata com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml in local (C:\Users\user\.m2\repository)
[DEBUG] Using transporter HttpTransporter with priority 5.0 for https://repo.maven.apache.org/maven2
[DEBUG] Using connector BasicRepositoryConnector with priority 0.0 for https://repo.maven.apache.org/maven2
Downloading from central: https://repo.maven.apache.org/maven2/com/akathist/maven/plugins/launch4j/launch4j-maven-plugin/maven-metadata.xml
Progress (1): maven-metadata.xml (1.4/1.6 kB)
Progress (1): maven-metadata.xml (1.6 kB)    

Downloaded from central: https://repo.maven.apache.org/maven2/com/akathist/maven/plugins/launch4j/launch4j-maven-plugin/maven-metadata.xml (1.6 kB at 4.1 kB/s)
[DEBUG] Writing tracking file 'C:\Users\user\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin\resolver-status.properties'
[DEBUG] Resolving artifact com.akathist.maven.plugins.launch4j:launch4j-maven-plugin:pom:2.4.1 from [central (https://repo.maven.apache.org/maven2, default, releases)]
[DEBUG] Using transporter HttpTransporter with priority 5.0 for https://repo.maven.apache.org/maven2
[DEBUG] Using connector BasicRepositoryConnector with priority 0.0 for https://repo.maven.apache.org/maven2
Downloading from central: https://repo.maven.apache.org/maven2/com/akathist/maven/plugins/launch4j/launch4j-maven-plugin/2.4.1/launch4j-maven-plugin-2.4.1.pom
Progress (1): launch4j-maven-plugin-2.4.1.pom (1.4/12 kB)
Progress (1): launch4j-maven-plugin-2.4.1.pom (2.7/12 kB)
Progress (1): launch4j-maven-plugin-2.4.1.pom (4.1/12 kB)
Progress (1): launch4j-maven-plugin-2.4.1.pom (5.5/12 kB)
Progress (1): launch4j-maven-plugin-2.4.1.pom (6.9/12 kB)
Progress (1): launch4j-maven-plugin-2.4.1.pom (8.2/12 kB)
Progress (1): launch4j-maven-plugin-2.4.1.pom (9.6/12 kB)
Progress (1): launch4j-maven-plugin-2.4.1.pom (11/12 kB) 
Progress (1): launch4j-maven-plugin-2.4.1.pom (12/12 kB)
Progress (1): launch4j-maven-plugin-2.4.1.pom (12 kB)   

Downloaded from central: https://repo.maven.apache.org/maven2/com/akathist/maven/plugins/launch4j/launch4j-maven-plugin/2.4.1/launch4j-maven-plugin-2.4.1.pom (12 kB at 111 kB/s)
[DEBUG] Writing tracking file 'C:\Users\user\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin\2.4.1\_remote.repositories'
[DEBUG] Writing tracking file 'C:\Users\user\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin\2.4.1\launch4j-maven-plugin-2.4.1.pom.lastUpdated'
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].version=2.1.3
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].artifactVersion=2.4.1
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].effectiveVersion=2.1.3
[DEBUG] [com.akathist.maven.plugins.launch4j:launch4j-maven-plugin].specified=true
…
[INFO] The following plugin updates are available:
[INFO]   com.akathist.maven.plugins.launch4j:launch4j-maven-plugin  2.1.3 -> 2.4.1
[INFO]   maven-failsafe-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   maven-release-plugin ............................ 2.5.3 -> 3.0.0-M6
[INFO]   maven-site-plugin .............................. 3.12.1 -> 4.0.0-M3
[INFO]   maven-surefire-plugin .......................... 2.22.2 -> 3.0.0-M7
[INFO]   org.springframework.boot:spring-boot-maven-plugin .. 2.7.3 -> 3.0.5
garretwilson commented 1 year ago

So it would seem not to be the case that the resolver mechanism is only looking at resolver-status.properties. It seems to be looking at maven-metadata-central.xml.

cstamas commented 1 year ago

So, now we need the resolver-status.properties.bad content.

Or better: copy now "bad files" over newly created ones and repeat /w debug. If you have again bad result, we have it

cstamas commented 1 year ago

So it would seem not to be the case that the resolver mechanism is only looking at resolver-status.properties. It seems to be looking at maven-metadata-central.xml.

For existence,.yes. if exists, goes to resolver-status.properties to check is update needed. I thought you renamed properties only.,.

garretwilson commented 1 year ago

I thought you renamed properties only.,.

I performed several steps, so we could narrow this down. First I renamed the properties file. Then I renamed the metadata XML file.

But hold on … major update soon.

cstamas commented 1 year ago

So: if XML is present, resolver would go to properties file to check "when was file fetched from remote". So only properties file should be renamed, xml should be left intact.

If XML renamed, resolver will go fetch it as it is not present.

garretwilson commented 1 year ago

So, now we need the resolver-status.properties.bad content.

No! I'm trying to say it has nothing to do with resolver-status.properties. The issue is in maven-metadata-central.xml (and maybe its checksum sidecar). I know it doesn't sound believable to you because of what you saw in the code, but I can only tell you the results of my testing.

For fun and everyone's entertainment, I did the following:

And … the problem appears again.

Thus it appears that I can reproduce the problem by having only these two files in ~\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin:

maven-metadata-central.xml

<?xml version="1.0" encoding="UTF-8"?>
<metadata>
  <groupId>com.akathist.maven.plugins.launch4j</groupId>
  <artifactId>launch4j-maven-plugin</artifactId>
  <versioning>
    <latest>2.1.2</latest>
    <release>2.1.2</release>
    <versions>
      <version>1.5.2</version>
      <version>1.6</version>
      <version>1.7</version>
      <version>1.7.1</version>
      <version>1.7.2</version>
      <version>1.7.3</version>
      <version>1.7.4</version>
      <version>1.7.5</version>
      <version>1.7.6</version>
      <version>1.7.7</version>
      <version>1.7.8</version>
      <version>1.7.9</version>
      <version>1.7.10</version>
      <version>1.7.11</version>
      <version>1.7.12</version>
      <version>1.7.13</version>
      <version>1.7.14</version>
      <version>1.7.15</version>
      <version>1.7.16</version>
      <version>1.7.17</version>
      <version>1.7.18</version>
      <version>1.7.19</version>
      <version>1.7.20</version>
      <version>1.7.21</version>
      <version>1.7.22</version>
      <version>1.7.23</version>
      <version>1.7.24</version>
      <version>1.7.25</version>
      <version>2.0.0</version>
      <version>2.0.1</version>
      <version>2.1.0</version>
      <version>2.1.1</version>
      <version>2.1.2</version>
    </versions>
    <lastUpdated>20210818055556</lastUpdated>
  </versioning>
</metadata>

maven-metadata-central.xml.sha1

8822fe33592659d2e93a8b8bd82ebb0b5ade46e2

Try putting only those two files in ~\.m2\repository\com\akathist\maven\plugins\launch4j\launch4j-maven-plugin. (Make sure to delete any later-version subdirectories, e.g. 2.4.1.) On my machine, that reproduces the problem. That is, the debug logs say, "Skipped remote request for com.akathist.maven.plugins.launch4j:launch4j-maven-plugin/maven-metadata.xml, locally cached metadata up-to-date".

garretwilson commented 1 year ago

So: if XML is present, resolver would go to properties file to check "when was file fetched from remote". So only properties file should be renamed, xml should be left intact.

I'm saying I did exactly that above. Nevertheless Maven did not go contact Maven Central, and it did not regenerate the properties file.

cstamas commented 1 year ago

Now you made me to get out of bed... 😄

garretwilson commented 1 year ago

Now you made me to get out of bed... 😄

Sorry!!! 😬 I was going to wait until tomorrow, but when you clarified that lastUpdated was a property, not a file extension, I though, what the heck, I spent the whole morning on this, I might as well keep going all afternoon. 😅

If you want to wait until tomorrow I completely understand. But if you keep replying I'll keep plodding forward in the testing. 😂

garretwilson commented 1 year ago

My best "hunch" guess at the moment based upon this behavior (and without knowing anything about the code other than what you've told me) is that either A) something in the code is accidentally bypassing the resolver-status.properties check, or B) somehow a path got mis-calculated and it's checking the resolver-status.properties of some other artifact (or something else altogether).

cstamas commented 1 year ago

Am trying to reproduce, but on laptop (not on desktop in office, where I had your setup), give me a five minutes or so...

cstamas commented 1 year ago

Damn me, I think I got it

garretwilson commented 1 year ago

Damn me, I think I got it

The suspense here is unbearable … haha

cstamas commented 1 year ago

full session https://gist.github.com/cstamas/f12f15ecc399022be2f2957a16fc0e11

step1: https://gist.github.com/cstamas/f12f15ecc399022be2f2957a16fc0e11#file-gistfile1-txt-L1 -- full build with empty local repo (snipped out the many screens of downloads) outcome: correct https://gist.github.com/cstamas/f12f15ecc399022be2f2957a16fc0e11#file-gistfile1-txt-L20

step2: https://gist.github.com/cstamas/f12f15ecc399022be2f2957a16fc0e11#file-gistfile1-txt-L54 Showing the contents of directories

step3: https://gist.github.com/cstamas/f12f15ecc399022be2f2957a16fc0e11#file-gistfile1-txt-L71 edited XML set content from here https://github.com/mojohaus/versions/issues/959#issuecomment-1563479777

step4: https://gist.github.com/cstamas/f12f15ecc399022be2f2957a16fc0e11#file-gistfile1-txt-L72 deleting properties file

step5: https://gist.github.com/cstamas/f12f15ecc399022be2f2957a16fc0e11#file-gistfile1-txt-L73 rinse repeat, invoke maven

outcome: https://gist.github.com/cstamas/f12f15ecc399022be2f2957a16fc0e11#file-gistfile1-txt-L93 NO update for launch4j-maven-plugin reported.

:beer:

garretwilson commented 1 year ago

NO update for launch4j-maven-plugin reported.

Just to confirm: you reproduced this?

cstamas commented 1 year ago

yes, with local empty repo (first run) 2.4.1 reported as latest (as expected), with altered local repo (2nd run) no update reported.

At first sight, this may be related to XML file timestamp....

After reproducing, I have this:

[cstamas@blondie versions-959]$ ll local/com/akathist/maven/plugins/launch4j/launch4j-maven-plugin/
total 8
drwxr-xr-x. 1 cstamas cstamas  174 máj   25 22.50 2.1.2
drwxr-xr-x. 1 cstamas cstamas  174 máj   25 22.48 2.1.3
drwxr-xr-x. 1 cstamas cstamas  174 máj   25 22.48 2.4.1
-rw-r--r--. 1 cstamas cstamas 1367 máj   25 22.50 maven-metadata-central.xml
-rw-r--r--. 1 cstamas cstamas   40 máj   25 22.48 maven-metadata-central.xml.sha1
[cstamas@blondie versions-959]$

Despite 2.4.1 version present, it goes unnoticed, as XML file (content copied from https://github.com/mojohaus/versions/issues/959#issuecomment-1563479777) does not contains it.

cstamas commented 1 year ago

Seems nope: changed TS to "2 days ago", no change. https://gist.github.com/cstamas/5aec55f97bed367e909dd685a073ea20

cstamas commented 1 year ago

So wow: this bug is a nasty one: it seems in this case, Maven metadata is handled "like any other artifact in release repository" -- with update policy "never"! This "never update" is true for release artifacts, but NOT for release metadata, that may contain changes (like new versions)... debugged so far, but now am now done.

We have a reproducer, and we have where to go from here.

In short, problem lies here: https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.10/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/DefaultUpdateCheckManager.java#L274

The check.policy() is "never", as central repo is release repository (artifacts are immutable), so the if receives false and goes to next block: https://github.com/apache/maven-resolver/blob/maven-resolver-1.9.10/maven-resolver-impl/src/main/java/org/eclipse/aether/internal/impl/DefaultUpdateCheckManager.java#L277

cstamas commented 1 year ago

Created https://issues.apache.org/jira/browse/MRESOLVER-363 This is a resolver bug, nothing to do with versions plugin. So if you agree, please close this issue.

garretwilson commented 1 year ago

So in Maven 3.9.1 this is happening to everyone who already has metadata? Some particular update to Maven Resolver basically left current metadata "locked" at its present state? Am I understanding that correctly?

Is the reason that you could not reproduce this earlier the fact that you had no metadata at all, and it was updating everything from scratch? But the moment anyone has metadata, it gets "locked" going forward, is that right?

cstamas commented 1 year ago

yes, with empty repo all is nicely downloaded.

also, many user (and even myself) in "daily routine" use this plugin with -U, so never notice: https://gist.github.com/cstamas/c22061e0c1f67961e8530fa056e609c4

garretwilson commented 1 year ago

So -U is a workaround that will force an update in this case for all my artifacts? I was reading on several Stack Overflow answers that -U only applies to -SNAPSHOT releases.

cstamas commented 1 year ago

Yes, -U is a workaround, and after this, my guess is it simply "stick" to users as w/o -U it MAY not report new things (just like in this issue case).

mvn -h says

 -U,--update-snapshots                  Forces a check for missing
                                        releases and updated snapshots on
                                        remote repositories

So is kinda badly worded. Without it this IF 1st block is executed, with it 2nd block: https://github.com/apache/maven/blob/maven-3.9.1/maven-core/src/main/java/org/apache/maven/internal/aether/DefaultRepositorySystemSessionFactory.java#L165

And my guess is that this "never" we see in case of metadata. Need to look more, as to me this "smells" like artifact policy (that is for release rightfully "never") is by mistake applied to metadata.

But now am really off.