jbangdev / jbang

Unleash the power of Java - JBang Lets Students, Educators and Professional Developers create, edit and run self-contained source-only Java programs with unprecedented ease.
https://jbang.dev
MIT License
1.43k stars 158 forks source link

Dependencies are not resolved when pom used as compile dependency commonly done by gradle published pom.xml #1065

Open edewit opened 2 years ago

edewit commented 2 years ago

I have the following script but for some reason jbang doesn't resolve the dependencies:

 edewit@localhost  ~/workspace/minecraft  jbang run --verbose server.java
[jbang] jbang version 0.83.0
[jbang] System Java version detected as 17
[jbang] System Java version matches requested version 17
[jbang] Resolving dependencies...
[jbang]     Resolving com.github.Minestom:Minestom:master-SNAPSHOT...[jbang] Deleting folder /home/edewit/.jbang/cache/jars/server.java.1daf3c6033a38e859e669d8040a68337bdbca312ed2f904937788c9ad8a6eefd.jar.tmp
[jbang] [ERROR] Could not resolve dependency com.github.Minestom:Minestom:master-SNAPSHOT
dev.jbang.cli.ExitException: Could not resolve dependency com.github.Minestom:Minestom:master-SNAPSHOT
    at dev.jbang.dependencies.DependencyUtil.lambda$resolveDependenciesViaAether$3(DependencyUtil.java:201)
    at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
    at dev.jbang.dependencies.DependencyUtil.resolveDependenciesViaAether(DependencyUtil.java:208)
    at dev.jbang.dependencies.DependencyUtil.resolveDependencies(DependencyUtil.java:113)
    at dev.jbang.dependencies.DependencyUtil.resolveDependencies(DependencyUtil.java:62)
    at dev.jbang.source.ScriptSource.resolveClassPath(ScriptSource.java:207)
    at dev.jbang.source.RunContext.resolveClassPath(RunContext.java:303)
    at dev.jbang.cli.BaseBuildCommand.buildJar(BaseBuildCommand.java:147)
    at dev.jbang.cli.BaseBuildCommand.build(BaseBuildCommand.java:122)
    at dev.jbang.cli.BaseBuildCommand.buildIfNeeded(BaseBuildCommand.java:86)
    at dev.jbang.cli.Run.prepareArtifacts(Run.java:99)
    at dev.jbang.cli.Run.doCall(Run.java:76)
    at dev.jbang.cli.BaseCommand.call(BaseCommand.java:85)
    at dev.jbang.cli.BaseCommand.call(BaseCommand.java:12)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
    at picocli.CommandLine.access$1300(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
    at dev.jbang.cli.JBang$3.handle(JBang.java:135)
    at dev.jbang.cli.JBang$3.handle(JBang.java:130)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
    at picocli.CommandLine.execute(CommandLine.java:2078)
    at dev.jbang.Main.main(Main.java:14)
Caused by: org.jboss.shrinkwrap.resolver.api.NoResolvedResultException: Unable to collect/resolve dependency tree for a resolution due to: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/), caused by: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.wrapException(MavenWorkingSessionImpl.java:503)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.resolveDependencies(MavenWorkingSessionImpl.java:242)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.using(MavenStrategyStageBaseImpl.java:70)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.withTransitivity(MavenStrategyStageBaseImpl.java:52)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.withTransitivity(MavenStrategyStageBaseImpl.java:39)
    at dev.jbang.dependencies.DependencyUtil.lambda$resolveDependenciesViaAether$3(DependencyUtil.java:194)
    ... 28 more
Caused by: org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:357)
    at org.jboss.shrinkwrap.resolver.impl.maven.bootstrap.MavenRepositorySystem.resolveDependencies(MavenRepositorySystem.java:121)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.resolveDependencies(MavenWorkingSessionImpl.java:239)
    ... 32 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:424)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:229)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:340)
    ... 34 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in mavencentral (https://repo1.maven.org/maven2/)
    at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:48)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:369)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:262)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:499)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:401)
    ... 36 more
[jbang] If you believe this a bug in jbang open issue at https://github.com/jbangdev/jbang/issues

but org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 is definitely in maven central. To be sure I create a quick gradle project with similar dependencies:

repositories {
    // Use JCenter for resolving dependencies.
    jcenter()
    maven { url 'https://repo.spongepowered.org/maven' }
    maven { url 'https://jitpack.io' }
}

dependencies {
    // This dependency is used by the application.
    implementation 'com.github.Minestom:Minestom:master-SNAPSHOT'
}

and that works just fine...

maxandersen commented 2 years ago

yeah something weird is going on - ill need to debug it.

i can just say i can confirm the issue - best guess is that somehow mavencentral doesn't get properly registered ...but just a hunch.

maxandersen commented 2 years ago

org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 is a pom - any reason you have that as a compile dependency ?

maxandersen commented 2 years ago

let me rephrase - somehow "org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4" shows up as dependenncy which actually does not exist - "org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:pom:3.1.4" does though.

maxandersen commented 2 years ago

so I created a minimal pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>gd.wa</groupId>
    <artifactId>minimal-pom</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>minimal-pom</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>1.8</java.version>
    </properties>

    <repositories>
        <repository>
            <id>jitpack.io</id>
            <url>https://jitpack.io</url>
        </repository>
        <repository>
            <id>sponge</id>
            <url>https://repo.spongepowered.org/maven</url>
            </repository>
        </repositories>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
        <groupId>com.github.Minestom</groupId>
        <artifactId>Minestom</artifactId>
        <version>master-SNAPSHOT</version>
        </dependency>    
    </dependencies>
</project>

and when I do:

mvn dependency:list I get:

[ERROR] Failed to execute goal on project minimal-pom: Could not resolve dependencies for project gd.wa:minimal-pom:jar:1.0-SNAPSHOT: Could not find artifact org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:jar:3.1.4 in jitpack.io (https://jitpack.io) -> [Help 1]

so jbang is technically reporting correctly here - some metadata in the chain of deps is wrong/bad.

Now I need to figure out why this is not triggered in gradle ...nor when doing mvn compile.

maxandersen commented 2 years ago

The issue is this line: https://github.com/Minestom/DependencyGetter/blob/066247f91b7c90db846134b192fdcf1507855a27/build.gradle.kts#L20

i'm 99% sure that is not valid so when pom.xml data generated it becomes invalid.

`implementation("org.jboss.shrinkwrap.resolver:shrinkwrap-resolver-depchain:3.1.4@pom")` could be enough to fix it.

but in general what is happening is a gradle idiom leaking into maven world which is not valid afaics.

maxandersen commented 2 years ago

so yeah, DependencyGetter generates this pom in https://jitpack.io/com/github/Minestom/DependencyGetter/1.0.1/DependencyGetter-1.0.1.pom

<dependency>
      <groupId>org.jboss.shrinkwrap.resolver</groupId>
      <artifactId>shrinkwrap-resolver-depchain</artifactId>
      <version>3.1.4</version>
      <scope>runtime</scope>
    </dependency>

that dependency does NOT exist.

This does:

<dependency>
      <groupId>org.jboss.shrinkwrap.resolver</groupId>
      <artifactId>shrinkwrap-resolver-depchain</artifactId>
      <version>3.1.4</version>
     <type>pom</type>
      <scope>runtime</scope>
    </dependency>

but that would also be wrong as you would have a pom file on classpath. it should be something like:

<dependency>
      <groupId>org.jboss.shrinkwrap.resolver</groupId>
      <artifactId>shrinkwrap-resolver-depchain</artifactId>
      <version>${version.shrinkwrap.resolvers}</version>
      <scope>test</scope>
      <type>pom</type>
    </dependency>

which you get if you use the bom as an actual bom - see https://docs.gradle.org/current/userguide/platforms.html#sub:bom_import

maxandersen commented 2 years ago

PR opened on minestorm that seem to fix it https://github.com/Minestom/DependencyGetter/pull/4

linux-china commented 2 years ago

How about following style, bom as real Bill of Material and pom for dependencies import?

//DEPS io.quarkus:quarkus-bom:1.11.0.Final@bom
//DEPS org.codehaus.groovy:groovy-all:3.0.9@pom

ShrinkWrap supports to resolve pom as dependencies:

        final ConfigurableMavenResolverSystem resolver = Maven.configureResolver();
        File[] files = resolver.resolve("org.codehaus.groovy:groovy-all:pom:3.0.9").withTransitivity().asFile();
        for (File file : files) {
            System.out.println(file.getAbsolutePath());
        }
maxandersen commented 2 years ago

It only support one bom. Not multiple which is what needs fixing.

But that doesn't solve this problem as the issue here is in the metadata - the pom.xml published by gradle are wrong/incorrect.

maxandersen commented 2 years ago

oh i missed your bom / pom differences - interesting idea but separate from this issue which is about pom.xml metadata published on maven central being wrong. you can open that idea as separate but it can't use the qualifier of the maven artifact reference to make the difference as no artifact has @bom artifact type.

linux-china commented 2 years ago

For example: org.codehaus.groovy:groovy-all:3.0.9@pom could be converted to org.codehaus.groovy:groovy-all:pom:3.0.9 and ShrinkWrap can resolve it.

yes, io.quarkus:quarkus-bom:1.11.0.Final@bom also is converted to io.quarkus:quarkus-bom:pom:1.11.0.Final , and resolved by ShrinkWrap, but it is marked as BOM. For artifacts without version, these artifacts will be sent to PomEquippedResolveStage to get version from effective BOM, and finally all artifacts have versions, and they can be resolved successfully by ShrinkWrap

Maven.resolver().resolve("io.quarkus:quarkus-bom:pom:2.6.1.Final").withoutTransitivity().asFile();
File repositoryHome = new File(System.getProperty("user.home"), ".m2/repository");
File bomFile = new File(repositoryHome, "io/quarkus/quarkus-bom/2.6.1.Final/quarkus-bom-2.6.1.Final.pom");
PomEquippedResolveStage bomResolveStage = Maven.resolver().loadPomFromFile(bomFile).importCompileAndRuntimeDependencies();
MavenResolvedArtifact artifact = bomResolveStage.resolve("com.google.guava:guava").withoutTransitivity().asSingle(MavenResolvedArtifact.class);
assertThat(artifact.getCoordinate().getVersion()).isEqualTo("30.1.1-jre");
maxandersen commented 2 years ago

Please open other issue for this. We are not going to be able change the semantics for how pom.xml works.

Your idea about allowing jbang users to express in jbang file to fetch dependencies inside a pom we can look at but this issue here is about what to do with the invalid pom.xml files published by gradle. Different issue.