Kotlin / kotlin-jupyter

Kotlin kernel for Jupyter/IPython
Apache License 2.0
1.1k stars 106 forks source link

Resolution errors with snapshot repo for dependency with classifier #285

Closed rnett closed 3 years ago

rnett commented 3 years ago

I have a library in maven local with a snapshot version, that depends on a another library with a snapshot version (added in the integration). The second library is also present in https://oss.sonatype.org/content/repositories/snapshots/. I'm seeing errors like:

@file:DependsOn("org.tensorflow:tensorflow-core-kotlin-jupyter:0.4.0-SNAPSHOT")
The problem is found in one of the loaded libraries: check library imports, dependencies and repositories
Error compiling code:
@file:Repository("https://oss.sonatype.org/content/repositories/snapshots/")
@file:DependsOn("org.tensorflow:tensorflow-core-api:0.4.0-SNAPSHOT:windows-x86_64")
import org.tensorflow.*
import org.tensorflow.op.*
import org.tensorflow.op.kotlin.*
import org.tensorflow.types.*
import org.tensorflow.types.family.*
import org.tensorflow.ndarray.*
import org.tensorflow.ndarray.index.*

Errors:
Failed to resolve org.tensorflow:tensorflow-core-api:0.4.0-SNAPSHOT:windows-x86_64:
File 'org.tensorflow:tensorflow-core-api:0.4.0-SNAPSHOT:windows-x86_64' not found
org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.tensorflow:tensorflow-core-api:0.4.0-SNAPSHOT:windows-x86_64 in https___repo.maven.apache.org_maven2_ (https://repo.maven.apache.org/maven2/)

org.jetbrains.kotlinx.jupyter.exceptions.ReplLibraryException: The problem is found in one of the loaded libraries: check library imports, dependencies and repositories
    at org.jetbrains.kotlinx.jupyter.exceptions.ReplLibraryExceptionKt.rethrowAsLibraryException(ReplLibraryException.kt:26)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext.addLibrary(CellExecutorImpl.kt:120)
    at org.jetbrains.kotlinx.jupyter.libraries.LibrariesScanner.addLibrariesFromClassLoader(LibrariesScanner.kt:33)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:83)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$3.invoke(CellExecutorImpl.kt:82)
    at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll(logging.kt:41)
    at org.jetbrains.kotlinx.jupyter.config.LoggingKt.catchAll$default(logging.kt:40)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:82)
    at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:13)
    at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$eval$1.invoke(repl.kt:373)
    at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl$eval$1.invoke(repl.kt:363)
    at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withEvalContext(repl.kt:348)
    at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.eval(repl.kt:363)
    at org.jetbrains.kotlinx.jupyter.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:291)
    at org.jetbrains.kotlinx.jupyter.ProtocolKt$shellMessagesHandler$res$1.invoke(protocol.kt:290)
    at org.jetbrains.kotlinx.jupyter.JupyterConnection$runExecution$execThread$1.invoke(connection.kt:162)
    at org.jetbrains.kotlinx.jupyter.JupyterConnection$runExecution$execThread$1.invoke(connection.kt:160)
    at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)
Caused by: org.jetbrains.kotlinx.jupyter.exceptions.ReplCompilerException: Failed to resolve org.tensorflow:tensorflow-core-api:0.4.0-SNAPSHOT:windows-x86_64:
File 'org.tensorflow:tensorflow-core-api:0.4.0-SNAPSHOT:windows-x86_64' not found
org.eclipse.aether.resolution.DependencyResolutionException: Could not find artifact org.tensorflow:tensorflow-core-api:0.4.0-SNAPSHOT:windows-x86_64 in https___repo.maven.apache.org_maven2_ (https://repo.maven.apache.org/maven2/)
    at org.jetbrains.kotlinx.jupyter.repl.impl.JupyterCompilerImpl.compileSync(JupyterCompilerImpl.kt:171)
    at org.jetbrains.kotlinx.jupyter.repl.impl.InternalEvaluatorImpl.eval(InternalEvaluatorImpl.kt:80)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$result$1.invoke(CellExecutorImpl.kt:63)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$execute$1$result$1.invoke(CellExecutorImpl.kt:62)
    at org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl.withHost(repl.kt:538)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl.execute(CellExecutorImpl.kt:62)
    at org.jetbrains.kotlinx.jupyter.repl.CellExecutor$DefaultImpls.execute$default(CellExecutor.kt:13)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext.execute(CellExecutorImpl.kt:145)
    at org.jetbrains.kotlinx.jupyter.api.libraries.CodeExecution$toExecutionCallback$1.invoke(CodeExecution.kt:20)
    at org.jetbrains.kotlinx.jupyter.api.libraries.CodeExecution$toExecutionCallback$1.invoke(CodeExecution.kt:19)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext.execute(CellExecutorImpl.kt:167)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext.runChild(CellExecutorImpl.kt:116)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext.runChild(CellExecutorImpl.kt:112)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext.access$runChild(CellExecutorImpl.kt:106)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext$addLibrary$1.invoke(CellExecutorImpl.kt:121)
    at org.jetbrains.kotlinx.jupyter.repl.impl.CellExecutorImpl$ExecutionContext$addLibrary$1.invoke(CellExecutorImpl.kt:120)
    at org.jetbrains.kotlinx.jupyter.exceptions.ReplLibraryExceptionKt.rethrowAsLibraryException(ReplLibraryException.kt:24)
    ... 17 more

Note that as far as I can tell, neither of the set repositories (snapshot and local) were tried. The looked-for artifact exists, see https://oss.sonatype.org/content/repositories/snapshots/org/tensorflow/tensorflow-core-api/0.4.0-SNAPSHOT/, and it exists in maven local. I can also use the fully resolved artifact id in a gradle project just fine.

My integration:


public class Integration : JupyterIntegration() {
  override fun Builder.onLoaded() {
    val os = System.getProperty("os.name").lowercase()
    val ext =
        when {
          os.contains("mac") -> "macosx-x86_64"
          os.startsWith("windows") -> "windows-x86_64"
          else -> "linux-x86_64"
        }

    val version = // gotten from resources

    if(version.lowercase().endsWith("snapshot")){
      repositories("https://oss.sonatype.org/content/repositories/snapshots/")
    }

    dependencies("org.tensorflow:tensorflow-core-api:$version:$ext")
    import(
        "org.tensorflow.*",
        "org.tensorflow.op.*",
        "org.tensorflow.op.kotlin.*",
        "org.tensorflow.types.*",
        "org.tensorflow.types.family.*",
        "org.tensorflow.ndarray.*",
        "org.tensorflow.ndarray.index.*"
    )
  }
}
rnett commented 3 years ago

It appears the classifier makes the difference, using org.tensorflow:tensorflow-core-api:$version instead works.

ileasile commented 3 years ago

All resolvers are a bit different in questions of coordinates formats. Maven resolver we're using now requires following format: <groupId>:<artifactId>[:<extension>[:<classifier>]]:<version> So, the following coordinates should work for you: org.tensorflow:tensorflow-core-api:jar:windows-x86_64:0.4.0-SNAPSHOT

ileasile commented 3 years ago

Tell me if it works for you

rnett commented 3 years ago

Wait, I forgot the jar, it works.