eclipse-equinox / equinox.bundles

Eclipse Public License 2.0
8 stars 16 forks source link

Incorrect `org.osgi.service.prefs` dependency in org.eclipse.equinox.preferences-3.10.0 #54

Closed JohnLBergqvist closed 2 years ago

JohnLBergqvist commented 2 years ago

The declaration for the org.osgi.service.prefs dependency inside https://repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.equinox.preferences/3.10.0/org.eclipse.equinox.preferences-3.10.0.pom

is wrong. Instead of being:

    <dependency>
      <groupId>org.osgi.service</groupId>
      <artifactId>org.osgi.service.prefs</artifactId>
      <version>[1.1.0,1.2.0)</version>
    </dependency>

as it currently is, it should be

    <dependency>
      <groupId>org.osgi</groupId>
      <artifactId>org.osgi.service.prefs</artifactId>
      <version>[1.1.0,1.2.0)</version>
    </dependency>

(basically, the <groupId> is wrong) as can be seen on Maven Central: https://search.maven.org/artifact/org.osgi/org.osgi.service.prefs/1.1.2/jar

Otherwise, I get a

org.eclipse.platform:org.eclipse.equinox.preferences:jar:3.10.0 -> org.osgi.service:org.osgi.service.prefs:jar:[1.1.0,1.2.0): No versions available for org.osgi.service:org.osgi.service.prefs:jar:[1.1.0,1.2.0) within specified range

error. If I patch the POM file in my local maven cache, this problem is resolved and the dependency downloads as expected.

HannesWell commented 2 years ago

Thank you for your report. The pom-file is generated during the release process (at least org.eclipse.equinox.preferences has none in git) and it looks like this generation is flawed. @akurtakov or @mickaelistria can you help here or know who can do better?

Fubuchi commented 2 years ago

We have the same issue today, the whole CI pipeline is broken because the group Id isn't existed. The current workaround is creating the folder structure as same as the wrong group Id and throwing the jar to there.

akurtakov commented 2 years ago

@sravanlakkimsetti You're the best person to help here. What/where/how generates the pom files deployed?

cdietrich commented 2 years ago

i also wonder if we can create a new repo in eclipses nexus where some milestones or nightlies for these and also platform and jdt maven artifacts can be published. e.g. xtext could use these in their integ tests so that problems like this one are detected earlier

laeubi commented 2 years ago

i also wonder if we can create a new repo in eclipses nexus

Why a new repo? Is the existing one not sufficient?

where some milestones or nightlies for these and also platform and jdt maven artifacts can be published. e.g. xtext could use these in their integ tests so that problems like this one are detected earlier

The problem is that current maven deployment is a nasty script witch need manual execution... One the equinox repos are merged I plan to try setup a more automated flow for the equinox that could be used as a basis for other repos as well.

cdietrich commented 2 years ago

when broken milestones are published to a release repo, then of course the "maven central" is broken problem would turn into the "eclipse release repo is broken"

laeubi commented 2 years ago

At least in tycho, staging or milestone repos has been proven to produce more hassle than they solve.

If we publish something "broken" we simply should fix that and publish a new version immediately.

akurtakov commented 2 years ago

repo.eclipse.org can be used to deploy SNAPSHOT versions and only builds that explicitly enable it will use them. We do that with Tycho and it works well sothere is no need for new repo. It's entirely separate question the effort to setup/automate that publishiing .

sravanlakkimsetti commented 2 years ago

I am trying a solution for this one. It will take some time. but hopefully by today or tomorrow I will be able to fix this one

castielfang commented 2 years ago

firstly maybe you should remove the version 3.10.0 from repository,stop to effect the more users

akurtakov commented 2 years ago

Can you actually remove things from Maven Central? I thought this is a not allowed.

laeubi commented 2 years ago

Can you actually remove things from Maven Central? I thought this is a not allowed.

https://stackoverflow.com/questions/9789611/removing-an-artifact-from-maven-central

You can only contact the support, but as no one is forced to upgrade their pom, you can simply use the old version in the meanwhile...

kesslerd commented 2 years ago

Can you actually remove things from Maven Central? I thought this is a not allowed.

https://stackoverflow.com/questions/9789611/removing-an-artifact-from-maven-central

You can only contact the support, but as no one is forced to upgrade their pom, you can simply use the old version in the meanwhile...

I have a transitive dependency on this library and the version I used for the main library is fixed. However, the dependency specifies a range of allowed version and the latest will be used. So I was forced to add a gradle dependency constraint

    constraints {
        implementation('org.eclipse.platform:org.eclipse.equinox.preferences:3.9.0') {
            force true
            because "Transitive dependency of org.eclipse.jdt:org.eclipse.jdt.core:3.17.0 newer version of org.eclipse.equinox.preferences points to wrong org.osgi location"
        }
    }
JohnLBergqvist commented 2 years ago

Can you actually remove things from Maven Central? I thought this is a not allowed.

https://stackoverflow.com/questions/9789611/removing-an-artifact-from-maven-central

You can only contact the support, but as no one is forced to upgrade their pom, you can simply use the old version in the meanwhile...

I also have a transitive dependency which is bringing this version in.

laeubi commented 2 years ago

I have a transitive dependency on this library and the version I used for the main library is fixed. However, the dependency specifies a range of allowed version and the latest will be used. So I was forced to add a gradle dependency constraint

Well that's what these features are for... you probably then should better blame the dependency specifying a version range without actually testing that the range is satiable...

kailbin commented 2 years ago

We have the same issue today, the whole CI pipeline is broken because the group Id isn't existed. The current workaround is creating the folder structure as same as the wrong group Id and throwing the jar to there.

So Do We !!

cdietrich commented 2 years ago

@laeubi all of eclipse platform and also emf have "ranges of death" e.g. from org.eclipse.core.runtime https://search.maven.org/artifact/org.eclipse.platform/org.eclipse.core.runtime/3.25.0/jar

    <dependency>
      <groupId>org.eclipse.platform</groupId>
      <artifactId>org.eclipse.equinox.preferences</artifactId>
      <version>[3.10.0,4.0.0)</version>
    </dependency>

this leads to always new transitive stuff beeing pulled

mcollovati commented 2 years ago

Can artifact relocation help in this case? https://maven.apache.org/guides/mini/guide-relocation.html

akurtakov commented 2 years ago

Can artifact relocation help in this case? https://maven.apache.org/guides/mini/guide-relocation.html

No, this is for when groupId actually changed. In this case we have wrong pom . We are working to upload fixed artifact and help to improve the way poms are generated and uploaded is more than welcome. Trying to look for immediate bandaid workarounds is not something we should be even losing time on.

PranavPurwar commented 2 years ago

Any updates on when this (fixed) version would be released?

akurtakov commented 2 years ago

As fast as we can - hopefully, today.

sravanlakkimsetti commented 2 years ago

New version 3.10.1 has been published now. https://oss.sonatype.org/service/local/repositories/releases/content/org/eclipse/platform/org.eclipse.equinox.preferences/3.10.1/org.eclipse.equinox.preferences-3.10.1.pom

should get indexed shortly

PranavPurwar commented 2 years ago

A Huge Thanks

sravanlakkimsetti commented 2 years ago

its available in maven central now https://repo1.maven.org/maven2/org/eclipse/platform/org.eclipse.equinox.preferences/3.10.1/

Artur- commented 2 years ago

Sorry if I am just adding noise here but I would like to understand if this is supposed to work now.

If I create a pom.xml with a dependency on org.eclipse.platform/org.eclipse.core.runtime/3.25.0 and nothing else then running mvn dependency:tree will fail with

Failed to collect dependencies at org.eclipse.platform:org.eclipse.core.runtime:jar:3.25.0 -> org.eclipse.platform:org.eclipse.equinox.preferences:jar:3.10.0 -> org.osgi.service:org.osgi.service.prefs:jar:[1.1.0,1.2.0): No versions available for org.osgi.service:org.osgi.service.prefs:jar:[1.1.0,1.2.0) within specified range -> [Help 1]

So even if org.eclipse.platform:org.eclipse.equinox.preferences:jar 3.10.1 is out, the range of [3.10.0,4.0.0) in platform:org.eclipse.core.runtime is resolved to 3.10.0 and not 3.10.1 (Maven happily downloads both to the local .m2 cache but still wants to use 3.10.0)

Because of this, projects are still broken

Example pom here https://gist.github.com/Artur-/e71a91b5b6acedce37e0b8426b7cdcca

PranavPurwar commented 2 years ago

You need to add equinox preferences as a separate dependency to maven/gradle for the changes to work currently.

cdietrich commented 2 years ago

but should maven pull the newer preference version?

from core.runtime:

   <dependency>
      <groupId>org.eclipse.platform</groupId>
      <artifactId>org.eclipse.equinox.preferences</artifactId>
      <version>[3.10.0,4.0.0)</version>
    </dependency>

automatically? why does it still pick the old one? if no i consider this only fixed half way

the bad artifact then will disturb mavens dependency conflict resolution forever

stephan-herrmann commented 2 years ago

In previous situations I observed that Maven explores the entire tree of possible dependencies, i.e., all allowed versions. If one of them fails resolution of transitive dependencies the build may fail, disregarding the fact, that the broken artifact is not even used, due to a newer matching version. I'm not sure if this applies here.

akurtakov commented 2 years ago

It should pick the latest one AFAIK. There are things like mirror used by the build not synced yet, local artifact satisfying available, timeout period not passed before new check happens (-U helps here) and etc. Let's give it a bit more time .

laeubi commented 2 years ago

In previous situations I observed that Maven explores the entire tree of possible dependencies, i.e., all allowed versions. If one of them fails resolution of transitive dependencies the build may fail, disregarding the fact, that the broken artifact is not even used, due to a newer matching version. I'm not sure if this applies here.

Yep this happens here as one can easily check with running the example in an empty repository like this:

mvn dependency:tree -X -Dmaven.repo.local=/tmp/repo

https://docs.oracle.com/middleware/1212/core/MAVEN/maven_version.htm#MAVEN402

cdietrich commented 2 years ago

so this can be resolved only

laeubi commented 2 years ago

so this can be resolved only

* by removing the bad artifact

* by adding the bad missing artifact as a stub/redirection etc

* avoiding any conflict working with explicit dependency exclude transitive
laeubi commented 2 years ago

By the way, snippet to fix the problem could be like this:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>groupid</groupId>
    <artifactId>aid</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>org.eclipse.platform</groupId>
            <artifactId>org.eclipse.core.runtime</artifactId>
            <version>3.25.0</version>
            <exclusions>
                <exclusion>
                    <groupId>org.osgi.service</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
</project>
vehovsky commented 2 years ago

@laeubi so everyone who ever used dependency on org.eclipse.core.runtime will have to do similar exclude or add org.eclipse.equinox.preferences under dependencyManagement in version that does work? Come on..

ja6a-regnosys commented 2 years ago

What we did is to add a dependency to our own maven repo with the osgi library under the "incorrect" GAV (Group, Artifact, Version coordinate):

<dependency>
    <groupId>org.osgi.service</groupId>
    <artifactId>org.osgi.service.prefs</artifactId>
    <version>1.1.2</version>
</dependency>
laeubi commented 2 years ago

so everyone who ever used dependency on org.eclipse.core.runtime will have to do similar exclude or add org.eclipse.equinox.preferences under dependencyManagement in version that does work?

Feel free to ignore this, I just can help with the technical issue and explain / show 'maven ways' to solve such issue. If you think maven should be more graceful in such cases you can suggest this here: https://issues.apache.org/jira/projects/MNG/

beside that, everyone affected by this should simply use a version range that excludes the bad version in the future see: https://docs.oracle.com/middleware/1212/core/MAVEN/maven_version.htm#MAVEN402 so eclipse can do this on the next release.

vehovsky commented 2 years ago

@laeubi well Maven could be more graceful no doubt. But since it's not, it's time to admit this is ma major issue! Because even releasing a fixed version is not enough.

Thanks to your usage of version ranges many of our old releases are not repeatable. And we will be affected by this issue anytime we will need to release a patch.

The only solution that would not force anyone who ever used or will use org.eclipse.core.runtime dependency to do exclusions or any other shenanigans is to do what @ja6a-regnosys did, but deploy it to maven central.

<dependency>
    <groupId>org.osgi.service</groupId>
    <artifactId>org.osgi.service.prefs</artifactId>
    <version>1.1.2</version>
</dependency>
laeubi commented 2 years ago

Thanks to your usage of version ranges many of our old releases are not repeatable.

It wasn't me to add these ranges... but they are for a reason to replicate OSGi version ranges, anyways if you think anything is really bad these are all open source and everyone is welcome to participate and help employ better maven practices.

he only solution that would not force anyone who ever used or will use org.eclipse.core.runtime dependency to do exclusions or any other shenanigans is to do what @ja6a-regnosys did, but deploy it to maven central.

I don't think this will be allowed to deploy arbitrary content to central but who knows? Maybe it would be more suitable to ask the support to remove the offending versions from central, or do a relocation (not sure if this is supported by central) with a completely fresh release.

cdietrich commented 2 years ago

@laeubi for sure not. but maybe we can beg the osgi project todo so

vehovsky commented 2 years ago

I don't think this will be allowed to deploy arbitrary content to central but who knows? Maybe it would be more suitable to ask the support to remove the offending versions from central, or do a relocation (not sure if this is supported by central) with a completely fresh release.

Agreed. Removal will work, anyone who has the artifact already cached in their local binary repo will have to remove it as well (plus devs will have to remove from local maven repos), but that is a minor issue.

laeubi commented 2 years ago

anyone who has the artifact already cached in their local binary repo will have to remove it as well

Not if the deploy a relocation pom pointing to the "fixed" version, this is not only possible for group-id changes as far as I can see: https://maven.apache.org/ref/3.8.6/maven-model/maven.html#class_relocation

alexlitovsky commented 2 years ago

I'm sorry, but for someone using these as transitive dependencies, and having made no changes to the application, what exactly is the fix now? Because an application that used to build is all of a sudden not building.

johnoct-au commented 2 years ago

@alexlitovsky I believe the fix or (current workaround) is this:

alexlitovsky commented 2 years ago

@alexlitovsky I believe the fix or (current workaround) is this:

* [Incorrect `org.osgi.service.prefs` dependency in org.eclipse.equinox.preferences-3.10.0 #54 (comment)](https://github.com/eclipse-equinox/equinox.bundles/issues/54#issuecomment-1155979528)

Thank you for that, but that's not a fix and not a workable workaround. Manually modifying the local .m2 repos on dozens of build nodes is not a workable solution for a serious product.

Come eclipse folks, please let us know how we can get our applications back into a working state.

laeubi commented 2 years ago

See

for a " workable workaround" for a "serious" product...

And pleas don't forget so support the eclipse community by a sponsoring: https://www.eclipse.org/sponsor/

sravanlakkimsetti commented 2 years ago

@alexlitovsky I believe the fix or (current workaround) is this:

* [Incorrect `org.osgi.service.prefs` dependency in org.eclipse.equinox.preferences-3.10.0 #54 (comment)](https://github.com/eclipse-equinox/equinox.bundles/issues/54#issuecomment-1155979528)

Thank you for that, but that's not a fix and not a workable workaround. Manually modifying the local .m2 repos on dozens of build nodes is not a workable solution for a serious product.

Come eclipse folks, please let us know how we can get our applications back into a working state.

We have deployed newer versions of the artifacts to maven. Please let us know how we can make maven to use latest versions during the build.

vehovsky commented 2 years ago

@alexlitovsky one solution using exclude was offered by @laeubi here

The other I mentioned above would leverage dependencyManagement, which I guess would be preferred for larger projects with transitive dependencies, which ideally import same BOM, so single change would fix the compilation for all your projects.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.eclipse.platform</groupId>
            <artifactId>org.eclipse.equinox.preferences</artifactId>
            <version>3.9.100</version>
        </dependency>
    </dependencies>
</dependencyManagement>

Still for me this is no more than temporary workaround. And I hope eclipse folks will contact maven central support and remove the broken release or deploy org.osgi.service.prefs under the GAV expected by the broken release.

alexlitovsky commented 2 years ago

@alexlitovsky one solution using exclude was offered by @laeubi here

The other I mentioned above would leverage dependencyManagement, which I guess would be preferred for larger projects with transitive dependencies, which ideally import same BOM, so single change would fix the compilation for all your projects.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.eclipse.platform</groupId>
            <artifactId>org.eclipse.equinox.preferences</artifactId>
            <version>3.9.100</version>
        </dependency>
    </dependencies>
</dependencyManagement>

Still for me this is no more than temporary workaround. And I hope eclipse folks will contact maven central support and remove the broken release or deploy org.osgi.service.prefs under the GAV expected by the broken release.

Thanks @vehovsky . We did fix this issue by uploading org.osgi.service.prefs under the wrong groupId to our mirror'd repo.

Hopefully this incident demonstrates why it's a terrible idea to use floating dependency versions.

bjhargrave commented 2 years ago

deploy org.osgi.service.prefs under the GAV expected by the broken release.

I will say this is not a possibility. OSGi is not interested in doing this to address this mistake.

vehovsky commented 2 years ago

I will say this is not a possibility. OSGi is not interested in doing this to address this mistake.

@bjhargrave Fair enough. So how is OSGi planning to address this mistake?