De-listed Maven Central versions

Fiouz commented 2 years ago

Description

Versions previously listed in Maven Central artifacts metadata (maven-metadata.xml) are no longer listed, breaking workflows that rely on automatically identifying available versions (with or without locking).

Examples:

spring-cloud-stream-dependencies
- Available 3.1.* versions: 3.1.0, 3.1.1, 3.1.2, 3.1.3, 3.1.4, 3.1.5
- 3.1.* versions listed in maven-metadata.xml: 3.1.0, 3.1.1
spring-cloud-bus-dependencies
- Available 3.0.* versions: 3.0.0, 3.0.1, 3.0.2, 3.0.3
- 3.0.* versions listed in maven-metadata.xml: 3.0.0, 3.0.1

Sample

build.gradle.kts

plugins {
    `java-library`
}

repositories {
    mavenCentral()
}

dependencies {
    implementation(platform("org.springframework.cloud:spring-cloud-stream-dependencies:[3.1.2,3.2)"))
    implementation("org.springframework.cloud:spring-cloud-starter-stream-kafka")
}

tasks {
    wrapper {
        gradleVersion = "7.3.1"
    }
}

gradle build output:

> Task :compileJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Could not resolve all files for configuration ':compileClasspath'.
   > Could not find any version that matches org.springframework.cloud:spring-cloud-stream-dependencies:[3.1.2,3.2).
     Versions that do not match:
       - 3.2.1
       - 3.2.0
       - 3.1.1
       - 3.1.0
       - 3.0.3.RELEASE
       - + 41 more
     Searched in the following locations:
       - https://repo.maven.apache.org/maven2/org/springframework/cloud/spring-cloud-stream-dependencies/maven-metadata.xml
     Required by:
         project :
   > Could not find org.springframework.cloud:spring-cloud-starter-stream-kafka:.
     Required by:
         project :

Workaround: switch to statically defined version / manual version locking

References

https://docs.gradle.org/current/userguide/single_versions.html https://maven.apache.org/pom.html#Dependency_Version_Requirement_Specification

marcingrzejszczak commented 2 years ago

Why is this issue filed here? I don't understand the problem.

Fiouz commented 2 years ago

The issue is filled here as

It seems widespread among Spring Cloud subprojects, so it led me to try to identify the common element regarding publishing to Maven Central (instead of filling one bug per individual Spring Cloud subproject)
I naively assumed that publishing would be coupled with any process/tooling related to releasing
This spring-cloud-release-tools subproject advertises itself as Tools used for the Spring Cloud release process, so it seemed a good candidate

If this issue tracker is not suitable due to my wrong assumptions/understanding, please let me know the relevant issue tracker, thank you.

The problem is that, the latest updates to maven-metadata.xml on many (if not all Spring Cloud subprojects) broke the assumption that released artifacts that were once available+discoverable will always stay available+discoverable from Maven Central (some of them are no longer discoverable).

In practice:

I had a project where the dependency was expressed as org.springframework.cloud:spring-cloud-stream-dependencies:3.1.+
At the time (a few months ago), Gradle resolved the version to 3.1.3, and all went well
More recently (December), I made Gradle refresh the version, and it suddenly reverted to 3.1.1 (because it is now the latest 3.1.* version listed in maven-metadata.xml), causing issues due to version regression

spencergibb commented 2 years ago

Moving to spring-cloud-build as spring-cloud-release-tools is some automation for us, it doesn't have anything to do with maven besides just running ./mvnw.

I still don't know what the issue is or how we could fix it. I think that file is generated by nexus by sontatype where we deploy, not by anything we control.

Fiouz commented 2 years ago

From past experiences with maven-deploy-plugin with private repositories (also using Sonatype Nexus), the Maven plugin itself will first download maven-metadata.xml, in order to upload an updated version which includes the artifact being published (in addition to actually uploading the main artifact). In other words, the repository is initially not involved into generating the maven-metadata.xml content, unless specifically requested to (Nexus has "Maven metadata repair" feature).

Unfortunately, I have no experience with Maven Central publishing, so I don't know whether that knowledge applies to it.

Fiouz commented 2 years ago

At its core, the issue is:

spring-cloud-stream-dependencies 3.1.3 does exist (as seen here or here)
So it should be listed under maven-metadata.xml, but it is not

(I also observed similar situation with other Spring Cloud subprojects)

spencergibb commented 2 years ago

I still don't think there's anything we can do.

Fiouz commented 2 years ago

Should I request Maven Central to repair Spring Cloud artifacts metadata?

My concern is that:

I cannot claim ownership of the affected artifacts
I will not be able entertain their reply if they challenge me about how those artifacts were published, as I don't know the intrinsic of Spring Cloud releasing (which credentials, which tool, which plugins, which version, what configuration, what changed, logs of the previous publications/uploads, etc.)

spencergibb commented 2 years ago

We can make the request

spring-cloud / spring-cloud-build