jbangdev / jbang

Unleash the power of Java - JBang Lets Students, Educators and Professional Developers create, edit and run self-contained source-only Java programs with unprecedented ease.
https://jbang.dev
MIT License
1.38k stars 155 forks source link

Using links to Git sources as a dependency does not work #974

Open gastaldi opened 3 years ago

gastaldi commented 3 years ago

I am using JBang 0.78.0 and followed the Using links to Git sources guide to use a dependency from my GitHub repository, like in the following:

///usr/bin/env jbang "$0" "$@" ; exit $?
//DEPS info.picocli:picocli:4.6.1
//DEPS com.github.gastaldi:quarkus-registry-generator:HEAD-SNAPSHOT
//JAVA_OPTIONS "-Djava.util.logging.SimpleFormatter.format=%1$s [%4$s] %5$s%6$s%n"
//JAVA 11

However it fails with:

~/workspace/quarkus-extension-catalog (main ✘)✹ ᐅ jbang --verbose .github/workflows/deploy.java --working-directory=. -o ../registry.quarkus.io-static/maven/
[jbang] jbang version 0.78.0
[jbang] System Java version detected as 11
[jbang] System Java version matches requested version 11
[jbang] Resolving dependencies...
[jbang]     Resolving info.picocli:picocli:4.6.1...Done
[jbang]     Resolving com.github.gastaldi:quarkus-registry-generator:HEAD-SNAPSHOT...[jbang] Deleting folder /home/ggastald/.jbang/cache/jars/deploy.java.0a711ce8ca4a75cf2f57d2bd70714f0b570e7c0b2d0abe7717f1450287b4401c.jar.tmp
[jbang] [ERROR] Could not resolve dependency com.github.gastaldi:quarkus-registry-generator:HEAD-SNAPSHOT
dev.jbang.cli.ExitException: Could not resolve dependency com.github.gastaldi:quarkus-registry-generator:HEAD-SNAPSHOT
    at dev.jbang.dependencies.DependencyUtil.lambda$resolveDependenciesViaAether$3(DependencyUtil.java:201)
    at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
    at dev.jbang.dependencies.DependencyUtil.resolveDependenciesViaAether(DependencyUtil.java:208)
    at dev.jbang.dependencies.DependencyUtil.resolveDependencies(DependencyUtil.java:113)
    at dev.jbang.dependencies.DependencyUtil.resolveDependencies(DependencyUtil.java:62)
    at dev.jbang.source.ScriptSource.resolveClassPath(ScriptSource.java:210)
    at dev.jbang.source.RunContext.resolveClassPath(RunContext.java:261)
    at dev.jbang.cli.BaseBuildCommand.buildJar(BaseBuildCommand.java:147)
    at dev.jbang.cli.BaseBuildCommand.build(BaseBuildCommand.java:122)
    at dev.jbang.cli.BaseBuildCommand.buildIfNeeded(BaseBuildCommand.java:82)
    at dev.jbang.cli.Run.prepareArtifacts(Run.java:86)
    at dev.jbang.cli.Run.doCall(Run.java:76)
    at dev.jbang.cli.BaseCommand.call(BaseCommand.java:85)
    at dev.jbang.cli.BaseCommand.call(BaseCommand.java:12)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
    at picocli.CommandLine.access$1300(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
    at dev.jbang.cli.Jbang$3.handle(Jbang.java:135)
    at dev.jbang.cli.Jbang$3.handle(Jbang.java:130)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
    at picocli.CommandLine.execute(CommandLine.java:2078)
    at dev.jbang.Main.main(Main.java:14)
Caused by: org.jboss.shrinkwrap.resolver.api.NoResolvedResultException: Unable to collect/resolve dependency tree for a resolution due to: Could not find artifact com.github.gastaldi:quarkus-registry-generator:jar:HEAD-SNAPSHOT in mavencentral (https://repo1.maven.org/maven2/), caused by: Could not find artifact com.github.gastaldi:quarkus-registry-generator:jar:HEAD-SNAPSHOT in mavencentral (https://repo1.maven.org/maven2/)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.wrapException(MavenWorkingSessionImpl.java:503)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.resolveDependencies(MavenWorkingSessionImpl.java:242)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.using(MavenStrategyStageBaseImpl.java:70)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.withTransitivity(MavenStrategyStageBaseImpl.java:52)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenStrategyStageBaseImpl.withTransitivity(MavenStrategyStageBaseImpl.java:39)
    at dev.jbang.dependencies.DependencyUtil.lambda$resolveDependenciesViaAether$3(DependencyUtil.java:194)
    ... 28 more
Caused by: org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact com.github.gastaldi:quarkus-registry-generator:jar:HEAD-SNAPSHOT in mavencentral (https://repo1.maven.org/maven2/)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:357)
    at org.jboss.shrinkwrap.resolver.impl.maven.bootstrap.MavenRepositorySystem.resolveDependencies(MavenRepositorySystem.java:121)
    at org.jboss.shrinkwrap.resolver.impl.maven.MavenWorkingSessionImpl.resolveDependencies(MavenWorkingSessionImpl.java:239)
    ... 32 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not find artifact com.github.gastaldi:quarkus-registry-generator:jar:HEAD-SNAPSHOT in mavencentral (https://repo1.maven.org/maven2/)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:424)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:229)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:340)
    ... 34 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact com.github.gastaldi:quarkus-registry-generator:jar:HEAD-SNAPSHOT in mavencentral (https://repo1.maven.org/maven2/)
    at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:48)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:369)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:262)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:499)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:401)
    ... 36 more
[jbang] If you believe this a bug in jbang open issue at https://github.com/jbangdev/jbang/issues
gastaldi commented 3 years ago

I have also tried with

//DEPS https://github.com/gastaldi/quarkus-registry-generator

but got the same error:

Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact com.github.gastaldi:quarkus-registry-generator:jar:-SNAPSHOT in mavencentral (https://repo1.maven.org/maven2/)
quintesse commented 3 years ago

I can confirm at least that the same happens to me locally. It also seems we don't have any integration tests for this? So right now I have no idea if this is broken or if there's some problem with this particular code.

quintesse commented 3 years ago

Btw manually adding //REPOS jitpack mavencentral doesn't help.

quintesse commented 3 years ago

The artifact does exist on JitPack: https://jitpack.io/#gastaldi/quarkus-registry-generator/-SNAPSHOT

Edit: sorry, that's wrong, it does NOT exist. (The page always exists, but no build has been done)

quintesse commented 3 years ago

If I remove the MavenCentral repo (and the picocli dep) the error is:

Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact com.github.gastaldi:quarkus-registry-generator:jar:-SNAPSHOT in jitpack (https://jitpack.io/)
    at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:48)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:369)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:75)
        ...
quintesse commented 3 years ago

@gastaldi can you perhaps try a quick test by setting up a dummy project with this:

    <repositories>
        <repository>
            <id>jitpack.io</id>
            <url>https://jitpack.io</url>
        </repository>
    </repositories>

and

    <dependency>
        <groupId>com.github.gastaldi</groupId>
        <artifactId>quarkus-registry-generator</artifactId>
        <version>-SNAPSHOT</version>
    </dependency>

To see if that works? That would at least rule out any problems with the service itself...

gastaldi commented 3 years ago

@quintesse that fails with:

[ERROR] Failed to execute goal on project points: Could not resolve dependencies for project com.example:points:jar:1.0.0-SNAPSHOT: Could not find artifact com.github.gastaldi:quarkus-registry-generator:jar:-SNAPSHOT in jitpack.io (https://jitpack.io) -> [Help 1]
quintesse commented 3 years ago

So perhaps it's a problem with the service then? 🤔

maxandersen commented 3 years ago

com.github.gastaldi:quarkus-registry-generator:HEAD-SNAPSHOT is not the way jitpack names its builds.

Its com.github.gastaldi:quarkus-registry-generator:HEAD AND you then must add jitpack.io manually as a repo (i.e. //REPOS jitpack)

From sources references would be to use //DEPS https://github.com/gastaldi/quarkus-registry-generator but that doesn't work either for some reason. I have a feeling jitpack changed their approach so there is no longer a way to genericially refer to the "HEAD" build one must use master or main explicitly ;/

so you would need to use //DEPS https://github.com/gastaldi/quarkus-registry-generator/tree/main#:SNAPSHOT

which works but sucks as the whole idea was to not require to have to deal with exact versions before you really need it.

maxandersen commented 3 years ago

One thing I noticed is different from this repo and lets say jbangdev/jbang is that quarkus-registry-generator has a build listed named HEAD - I wonder if that somehow breaks HEAD-SNAPSHOT fetches on the jitpack.io side ? at least using com.github.jbangdev:jbang:HEAD-SNAPSHOT seem to work for me ?

maxandersen commented 3 years ago

correction, it seems like it works but eventually it fails too...so build is triggered but no longer seem to be able to complete/return.

I'm 98% convinced jitpack.io changed behaviour here. Unfortunately they don't seem to respond on gitter, twitter nor support lately but I've given it a go via their support/feedback from on their website.

maxandersen commented 3 years ago

compare: https://jitpack.io/com/github/gastaldi/quarkus-registry-generator/ with https://jitpack.io/com/github/jbangdev/jbang

koppor commented 10 months ago

Side note: The feature is documented at https://www.jbang.dev/documentation/guide/latest/dependencies.html#using-links-to-git-sources.

AND you then must add jitpack.io manually as a repo (i.e. //REPOS jitpack)

This needs to be added to the documentation


When just following the documentation:

I have following java file:

//DEPS https://github.com/koppor/heylogs/tree/add-git-diff-check
public class heylogs {
    public static void main(String... args) throws Exception {
        nbbrd.heylogs.cli.HeylogsCommand.main(args);
    }
}

jbang outputs:

> jbang .github\heylogs.java -g
Picked up JAVA_TOOL_OPTIONS: -Duser.language=en
[jbang] Resolving dependencies...
[jbang]    com.github.koppor:heylogs:add-git-diff-check
[jbang] [ERROR] Could not resolve dependencies: Could not find artifact com.github.koppor:heylogs:jar:add-git-diff-check in central (https://repo1.maven.org/maven2/)
[jbang] Run with --verbose for more details

When ading //REPOS jitpack

I have following java file:

//REPOS jitpack
//DEPS https://github.com/koppor/heylogs/tree/add-git-diff-check
public class heylogs {
    public static void main(String... args) throws Exception {
        nbbrd.heylogs.cli.HeylogsCommand.main(args);
    }
}

jbang outputs:

> jbang .github\heylogs.java -g
Picked up JAVA_TOOL_OPTIONS: -Duser.language=en
[jbang] Resolving dependencies...
[jbang]    com.github.koppor:heylogs:add-git-diff-check
[jbang] [ERROR] Could not resolve dependencies: Could not find artifact com.github.koppor:heylogs:jar:add-git-diff-check in jitpack (https://jitpack.io/)
[jbang] Run with --verbose for more details

All NOT working here.

I wonder why :jar is magically added to the dependency

maxandersen commented 10 months ago

:jar is the canonical naming of jar artifacts.

Jitpack is quite flakey. My guess is we are still hit with them no longer dealing with HEAD as uniform way to get latest main branch no matter if master or main used for name.

If that's the case add the branch name explicitly.

koppor commented 10 months ago

In my case: The constructed URL returns "Tag or commit 'add-git-diff-check' not found. Rechecking.".

I tried with the commit id, also not working.

I tried with a freshly created tag.

Now, I get

[jbang] [0:348] Resolving dependencies...
[jbang] [0:421]    com.github.koppor:heylogs:test
[jbang] [10:721]    org.junit:junit-bom:5.10.0
[jbang] [10:724]    com.github.nbbrd.java-service-util:java-service-bom:1.7.0
[jbang] [11:837] Deleting folder C:\Users\koppor\.jbang\cache\jars\heylogs.java.1ddf3d869d7764bab2321e907dc09c8e44f88abcc42347fbc7cb8d39aeef5395\classes
[jbang] [11:840] Deleting folder C:\Users\koppor\.jbang\cache\jars\heylogs.java.1ddf3d869d7764bab2321e907dc09c8e44f88abcc42347fbc7cb8d39aeef5395\generated
[jbang] [11:841] [ERROR] Could not resolve dependencies: Could not find artifact com.github.koppor:heylogs:jar:test in jitpack (https://jitpack.io/)
dev.jbang.cli.ExitException: Could not resolve dependencies: Could not find artifact com.github.koppor:heylogs:jar:test in jitpack (https://jitpack.io/)

That error message could be more detailed. Reason: it seems, jbang could resolve the root dependency, but has issues with some child dependencies... I would like to know, which one failed.

OK, the error is on my side, because I want to include a sub-project in jitpack, which is IMHO not possible. Update: It should be possible https://stackoverflow.com/a/48024281/873282

(working version shows it - https://github.com/nbbrd/jbang-catalog/blob/master/heylogs.java)


For the googlers some links:

I got it running w/ jitpack dependency. "RTFM of jitpack.io" is the answer for me.

jbang --repos jitpack,central -m nbbrd.heylogs.cli.HeylogsCommand com.github.koppor.heylogs:heylogs-cli:jitpack-SNAPSHOT --help

Not sure how to do with GitHub links and sub projects.