search

aws / aws-sdk-java-v2

The official AWS SDK for Java - Version 2
Apache License 2.0
2.16k stars 836 forks source link

java.lang.ClassNotFoundException: software.amazon.awssdk.core.runtime.transform.Unmarshaller #970

Closed oripwk closed 5 years ago

oripwk commented 5 years ago

Current Behavior

Getting exception

java.lang.ClassNotFoundException: software.amazon.awssdk.core.runtime.transform.Unmarshaller`

when using amazon-kinesis-client 2.0.5.

Stacktrace

java.lang.NoClassDefFoundError: software/amazon/awssdk/core/runtime/transform/Unmarshaller
    at software.amazon.awssdk.services.dynamodb.DefaultDynamoDbAsyncClientBuilder.buildClient(DefaultDynamoDbAsyncClientBuilder.java:28)
    at software.amazon.awssdk.services.dynamodb.DefaultDynamoDbAsyncClientBuilder.buildClient(DefaultDynamoDbAsyncClientBuilder.java:22)
    at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.build(SdkDefaultClientBuilder.java:118)
    at com.tg.mobile_events.common.KinesisSubscriber.<init>(KinesisSubscriber.kt:35)
    at com.tg.mobile_events.Dependencies$DefaultImpls.getKinesisSubscriber(Dependencies.kt:73)
    at com.tg.mobile_events.App.getKinesisSubscriber(App.kt:23)
    at com.tg.mobile_events.App.start$suspendImpl(App.kt:41)
    at com.tg.mobile_events.App$start$1.invokeSuspend(App.kt)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)
    at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:236)
    at io.vertx.kotlin.coroutines.VertxCoroutineExecutor.execute(VertxCoroutine.kt:215)
    at kotlinx.coroutines.ExecutorCoroutineDispatcherBase.dispatch(Executors.kt:64)
    at kotlinx.coroutines.DispatchedKt.dispatch(Dispatched.kt:259)
    at kotlinx.coroutines.AbstractContinuation.dispatchResume(AbstractContinuation.kt:182)
    at kotlinx.coroutines.AbstractContinuation.completeStateUpdate(AbstractContinuation.kt:250)
    at kotlinx.coroutines.AbstractContinuation.updateStateToFinal(AbstractContinuation.kt:222)
    at kotlinx.coroutines.AbstractContinuation.resumeImpl(AbstractContinuation.kt:195)
    at kotlinx.coroutines.AbstractContinuation.resumeWith(AbstractContinuation.kt:141)
    at io.vertx.kotlin.coroutines.VertxCoroutineKt$awaitEvent$2$1.handle(VertxCoroutine.kt:52)
    at io.vertx.config.impl.ConfigRetrieverImpl.lambda$getConfig$2(ConfigRetrieverImpl.java:180)
    at io.vertx.config.impl.ConfigRetrieverImpl.lambda$compute$9(ConfigRetrieverImpl.java:294)
    at io.vertx.core.impl.CompositeFutureImpl.lambda$all$1(CompositeFutureImpl.java:49)
    at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:125)
    at io.vertx.config.impl.ConfigRetrieverImpl.lambda$null$6(ConfigRetrieverImpl.java:264)
    at io.vertx.config.impl.ConfigurationProvider.lambda$null$0(ConfigurationProvider.java:83)
    at io.vertx.core.impl.ContextImpl.lambda$null$0(ContextImpl.java:284)
    at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:320)
    at io.vertx.core.impl.EventLoopContext.lambda$executeAsync$0(EventLoopContext.java:38)
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:466)
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: software.amazon.awssdk.core.runtime.transform.Unmarshaller
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 34 more

Classpath

ch.qos.logback:logback-classic:1.1.3
ch.qos.logback:logback-core:1.1.3
com.datadoghq:java-dogstatsd-client:2.6.1
com.fasterxml.jackson.core:jackson-annotations:2.9.6
com.fasterxml.jackson.core:jackson-core:2.9.7
com.fasterxml.jackson.core:jackson-databind:2.9.7
com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.9.6
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.7
com.fasterxml.jackson.datatype:jackson-datatype-guava:2.6.3
com.fasterxml.jackson.datatype:jackson-datatype-jdk7:2.6.3
com.fasterxml.jackson.datatype:jackson-datatype-joda:2.6.3
com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.9.7
com.fasterxml.jackson.module:jackson-module-afterburner:2.6.3
com.fasterxml.jackson.module:jackson-module-kotlin:2.9.7
com.fasterxml:classmate:1.1.0
com.github.jnr:jffi:1.2.15
com.github.jnr:jffi:native:1.2.15
com.github.jnr:jnr-constants:0.9.8
com.github.jnr:jnr-enxio:0.16
com.github.jnr:jnr-ffi:2.1.4
com.github.jnr:jnr-posix:3.0.35
com.github.jnr:jnr-unixsocket:0.18
com.github.jnr:jnr-x86asm:1.0.2
com.google.code.findbugs:jsr305:3.0.2
com.google.errorprone:error_prone_annotations:2.1.3
com.google.guava:guava:26.0-jre
com.google.j2objc:j2objc-annotations:1.1
com.google.protobuf:protobuf-java:2.6.1
com.kohlschutter.junixsocket:junixsocket-common:2.0.4
com.kohlschutter.junixsocket:junixsocket-native-common:2.0.4
com.tg:tg-core:2.07-SNAPSHOT
com.typesafe.netty:netty-reactive-streams-http:2.0.0
com.typesafe.netty:netty-reactive-streams:2.0.0
com.typesafe:config:1.3.0
com.zaxxer:HikariCP:3.2.0
commons-codec:commons-codec:1.10
commons-logging:commons-logging:1.2
io.dropwizard.metrics:metrics-core:4.0.2
io.dropwizard.metrics:metrics-jvm:4.0.2
io.dropwizard:dropwizard-jackson:0.9.1
io.dropwizard:dropwizard-lifecycle:0.9.1
io.dropwizard:dropwizard-metrics:0.9.1
io.dropwizard:dropwizard-util:0.9.1
io.dropwizard:dropwizard-validation:0.9.1
io.github.benas:random-beans:3.7.0
io.github.lukehutch:fast-classpath-scanner:2.2.1
io.github.microutils:kotlin-logging-common:1.6.20
io.github.microutils:kotlin-logging:1.6.20
io.mockk:mockk-agent-api:1.8.13
io.mockk:mockk-agent-common:1.8.13
io.mockk:mockk-agent-jvm:1.8.13
io.mockk:mockk-common:1.8.13
io.mockk:mockk-dsl-jvm:1.8.13
io.mockk:mockk-dsl:1.8.13
io.mockk:mockk:1.8.13
io.netty:netty-buffer:4.1.31.Final
io.netty:netty-codec-dns:4.1.30.Final
io.netty:netty-codec-http2:4.1.31.Final
io.netty:netty-codec-http:4.1.31.Final
io.netty:netty-codec-socks:4.1.30.Final
io.netty:netty-codec:4.1.31.Final
io.netty:netty-common:4.1.31.Final
io.netty:netty-handler-proxy:4.1.30.Final
io.netty:netty-handler:4.1.31.Final
io.netty:netty-resolver-dns:4.1.30.Final
io.netty:netty-resolver:4.1.31.Final
io.netty:netty-transport-native-epoll:linux-x86_64:4.1.31.Final
io.netty:netty-transport-native-unix-common:4.1.31.Final
io.netty:netty-transport:4.1.31.Final
io.reactivex.rxjava2:rxjava:2.1.14
io.vertx:vertx-auth-common:3.6.2
io.vertx:vertx-bridge-common:3.6.2
io.vertx:vertx-config-hocon:3.6.2
io.vertx:vertx-config:3.6.2
io.vertx:vertx-core:3.6.2
io.vertx:vertx-dropwizard-metrics:3.6.2
io.vertx:vertx-lang-kotlin-coroutines:3.6.2
io.vertx:vertx-lang-kotlin:3.6.2
io.vertx:vertx-web-client:3.6.2
io.vertx:vertx-web-common:3.6.2
io.vertx:vertx-web:3.6.2
javax.annotation:javax.annotation-api:1.3.1
javax.servlet:javax.servlet-api:3.1.0
javax.validation:validation-api:1.1.0.Final
javax.xml.bind:jaxb-api:2.2.12
javax.xml.bind:jaxb-api:2.3.0
joda-time:joda-time:2.9
junit:junit:4.12
mysql:mysql-connector-java:6.0.6
net.bytebuddy:byte-buddy-agent:1.8.22
net.bytebuddy:byte-buddy:1.8.22
net.java.dev.jna:jna-platform:4.5.1
net.java.dev.jna:jna:4.5.1
org.apache.commons:commons-compress:1.18
org.apache.commons:commons-lang3:3.7
org.apache.httpcomponents:fluent-hc:4.3.6
org.apache.httpcomponents:httpclient:4.5.6
org.apache.httpcomponents:httpcore:4.4.10
org.apache.logging.log4j:log4j-api:2.11.1
org.apache.logging.log4j:log4j-core:2.11.1
org.apache.logging.log4j:log4j-slf4j-impl:2.11.1
org.apiguardian:apiguardian-api:1.0.0
org.awaitility:awaitility:3.1.3
org.checkerframework:checker-qual:2.5.2
org.codehaus.mojo:animal-sniffer-annotations:1.14
org.coursera:dropwizard-metrics-datadog:1.1.14
org.coursera:metrics-datadog:1.1.14
org.eclipse.jetty:jetty-http:9.2.13.v20150730
org.eclipse.jetty:jetty-io:9.2.13.v20150730
org.eclipse.jetty:jetty-server:9.2.13.v20150730
org.eclipse.jetty:jetty-util:9.2.13.v20150730
org.flywaydb:flyway-core:5.2.1
org.glassfish:javax.el:3.0.0
org.hamcrest:hamcrest-core:1.3
org.hamcrest:hamcrest-library:1.3
org.hibernate:hibernate-validator:5.2.2.Final
org.jboss.logging:jboss-logging:3.2.1.Final
org.jetbrains.kotlin:kotlin-reflect:1.3.10
org.jetbrains.kotlin:kotlin-stdlib-common:1.3.11
org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.3.0
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.3.0
org.jetbrains.kotlin:kotlin-stdlib:1.3.11
org.jetbrains.kotlin:kotlin-test-common:1.3.11
org.jetbrains.kotlin:kotlin-test:1.3.11
org.jetbrains.kotlinx:kotlinx-coroutines-core-common:1.0.1
org.jetbrains.kotlinx:kotlinx-coroutines-core:1.0.1
org.jetbrains.kotlinx:kotlinx-coroutines-jdk8:1.0.1
org.jetbrains.kotlinx:kotlinx-coroutines-slf4j:1.0.1
org.jetbrains.spek:spek-api:1.2.1
org.jetbrains.spek:spek-junit-platform-engine:1.2.1
org.jetbrains:annotations:13.0
org.jetbrains:annotations:15.0
org.jooq:jooq:3.11.7
org.junit.platform:junit-platform-commons:1.3.1
org.junit.platform:junit-platform-engine:1.3.1
org.junit.platform:junit-platform-launcher:1.3.1
org.junit.platform:junit-platform-runner:1.3.1
org.junit.platform:junit-platform-suite-api:1.3.1
org.objenesis:objenesis:2.6
org.opentest4j:opentest4j:1.1.1
org.ow2.asm:asm-analysis:5.0.3
org.ow2.asm:asm-commons:5.0.3
org.ow2.asm:asm-tree:5.0.3
org.ow2.asm:asm-util:5.0.3
org.ow2.asm:asm:5.0.3
org.reactivestreams:reactive-streams:1.0.2
org.rnorth.duct-tape:duct-tape:1.0.7
org.rnorth.visible-assertions:visible-assertions:2.1.1
org.rnorth:tcp-unix-socket-proxy:1.0.2
org.scijava:native-lib-loader:2.0.2
org.slf4j:slf4j-api:1.7.25
org.slf4j:slf4j-ext:1.7.25
org.testcontainers:testcontainers:1.10.1
org.yaml:snakeyaml:1.23
software.amazon.awssdk:annotations:2.1.0
software.amazon.awssdk:apache-client:2.1.0
software.amazon.awssdk:auth:2.1.0
software.amazon.awssdk:aws-cbor-protocol:2.1.0
software.amazon.awssdk:aws-core:2.1.0
software.amazon.awssdk:aws-json-protocol:2.1.0
software.amazon.awssdk:aws-query-protocol:2.1.0
software.amazon.awssdk:cloudwatch:2.0.6
software.amazon.awssdk:dynamodb:2.0.6
software.amazon.awssdk:http-client-spi:2.1.0
software.amazon.awssdk:kinesis:2.1.0
software.amazon.awssdk:netty-nio-client:2.1.0
software.amazon.awssdk:profiles:2.1.0
software.amazon.awssdk:protocol-core:2.1.0
software.amazon.awssdk:regions:2.1.0
software.amazon.awssdk:sdk-core:2.1.0
software.amazon.awssdk:sqs:2.1.0
software.amazon.awssdk:utils:2.1.0
software.amazon.kinesis:amazon-kinesis-client:2.0.5
software.amazon:flow:1.7

Gradle dependencies

dependencies {
    compile(
            [group: 'org.jetbrains.kotlin', name: 'kotlin-stdlib'],
            [group: 'io.vertx', name: 'vertx-lang-kotlin', version: "${vertxVersion}"],
            [group: 'io.vertx', name: 'vertx-lang-kotlin-coroutines', version: "${vertxVersion}"],
            [group: 'io.vertx', name: 'vertx-web', version: "${vertxVersion}"],
            [group: 'io.vertx', name: 'vertx-config-hocon', version: "${vertxVersion}"],
            [group: 'io.vertx', name: 'vertx-dropwizard-metrics', version: "${vertxVersion}"],
            [group: 'io.vertx', name: 'vertx-web-client', version: "${vertxVersion}"],
            [group: 'com.fasterxml.jackson.module', name: 'jackson-module-kotlin', version: '2.9.7'],
            [group: 'mysql', name: 'mysql-connector-java', version: '6.0.6'],
            [group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: '2.11.1'],
            [group: 'org.coursera', name: 'dropwizard-metrics-datadog', version: '1.1.14'],
            [group: 'com.fasterxml.jackson.dataformat', name: 'jackson-dataformat-yaml', version: '2.9.7'],
            [group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-jsr310', version: '2.9.7'],
            [group: 'io.github.microutils', name: 'kotlin-logging', version: '1.6.10'],
            [group: 'org.jooq', name: 'jooq', version: '3.11.7'],
            [group: 'com.zaxxer', name: 'HikariCP', version: '3.2.0'],
            [group: 'com.tg', name: 'tg-core', version: '2.07-SNAPSHOT'],
            [group: 'software.amazon.kinesis', name: 'amazon-kinesis-client', version: '2.0.5'],
    )

    testCompile(
            [group: 'org.jetbrains.kotlin', name: 'kotlin-test'],
            [group: 'org.jetbrains.spek', name: 'spek-api', version: "${spekVersion}"],
            [group: 'org.junit.platform', name: 'junit-platform-runner', version: '1.3.1'],
            [group: 'io.mockk', name: 'mockk', version: '1.8.13'],
            [group: 'org.flywaydb', name: 'flyway-core', version: '5.2.1'],
            [group: 'org.testcontainers', name: 'testcontainers', version: '1.10.1'],
            [group: 'org.awaitility', name: 'awaitility', version: '3.1.3'],
            [group: 'io.github.benas', name: 'random-beans', version: '3.7.0']
    )

    testRuntime(
            [group: 'org.jetbrains.spek', name: 'spek-junit-platform-engine', version: "${spekVersion}"]
    )
}

Possible Solution

No possible solution

Context

I was trying to setup a Kinesis consumer with KCL

Your Environment

zoewangg commented 5 years ago

Apologies for the delay.

It seems odd because software.amazon.awssdk.core.runtime.transform.Unmarshaller does not exist in the GA version. Could you try to clean your local .m2 repo and IDE cache?

oripwk commented 5 years ago

Hi,

I'm running it inside a docker container. The same problem still happens.

We're really stuck and nothing seems to solve the issue.

This is the directory content of the classpath:

Oris-MacBook-Pro:~ orip$ docker exec my-app ls /opt/docker/lib
HikariCP-3.2.0.jar
amazon-kinesis-client-2.0.5.jar
animal-sniffer-annotations-1.14.jar
annotations-13.0.jar
annotations-2.1.0.jar
apache-client-2.1.0.jar
asm-5.0.3.jar
asm-analysis-5.0.3.jar
asm-commons-5.0.3.jar
asm-tree-5.0.3.jar
asm-util-5.0.3.jar
auth-2.1.0.jar
aws-cbor-protocol-2.1.0.jar
aws-core-2.1.0.jar
aws-json-protocol-2.1.0.jar
aws-query-protocol-2.1.0.jar
checker-qual-2.5.2.jar
classmate-1.1.0.jar
cloudwatch-2.0.6.jar
commons-codec-1.10.jar
commons-lang3-3.7.jar
commons-logging-1.2.jar
config-1.3.0.jar
dropwizard-jackson-0.9.1.jar
dropwizard-lifecycle-0.9.1.jar
dropwizard-metrics-0.9.1.jar
dropwizard-metrics-datadog-1.1.14.jar
dropwizard-util-0.9.1.jar
dropwizard-validation-0.9.1.jar
dynamodb-2.0.6.jar
error_prone_annotations-2.1.3.jar
flow-1.7.jar
fluent-hc-4.3.6.jar
guava-26.0-jre.jar
hibernate-validator-5.2.2.Final.jar
http-client-spi-2.1.0.jar
httpclient-4.5.6.jar
httpcore-4.4.10.jar
j2objc-annotations-1.1.jar
jackson-annotations-2.9.6.jar
jackson-core-2.9.7.jar
jackson-databind-2.9.7.jar
jackson-dataformat-cbor-2.9.6.jar
jackson-dataformat-yaml-2.9.7.jar
jackson-datatype-guava-2.6.3.jar
jackson-datatype-jdk7-2.6.3.jar
jackson-datatype-joda-2.6.3.jar
jackson-datatype-jsr310-2.9.7.jar
jackson-module-afterburner-2.6.3.jar
jackson-module-kotlin-2.9.7.jar
java-dogstatsd-client-2.6.1.jar
javax.el-3.0.0.jar
javax.servlet-api-3.1.0.jar
jaxb-api-2.2.12.jar
jboss-logging-3.2.1.Final.jar
jetty-http-9.2.13.v20150730.jar
jetty-io-9.2.13.v20150730.jar
jetty-server-9.2.13.v20150730.jar
jetty-util-9.2.13.v20150730.jar
jffi-1.2.15-native.jar
jffi-1.2.15.jar
jnr-constants-0.9.8.jar
jnr-enxio-0.16.jar
jnr-ffi-2.1.4.jar
jnr-posix-3.0.35.jar
jnr-unixsocket-0.18.jar
jnr-x86asm-1.0.2.jar
joda-time-2.9.jar
jooq-3.11.7.jar
jsr305-3.0.2.jar
kinesis-2.1.0.jar
kotlin-logging-1.6.20.jar
kotlin-logging-common-1.6.20.jar
kotlin-reflect-1.3.10.jar
kotlin-stdlib-1.3.11.jar
kotlin-stdlib-common-1.3.11.jar
kotlin-stdlib-jdk7-1.3.11.jar
kotlin-stdlib-jdk8-1.3.11.jar
kotlinx-coroutines-core-1.0.1.jar
kotlinx-coroutines-core-common-1.0.1.jar
kotlinx-coroutines-jdk8-1.0.1.jar
kotlinx-coroutines-slf4j-1.0.1.jar
log4j-api-2.11.1.jar
log4j-core-2.11.1.jar
log4j-slf4j-impl-2.11.1.jar
logback-classic-1.1.3.jar
logback-core-1.1.3.jar
metrics-core-4.0.2.jar
metrics-datadog-1.1.14.jar
metrics-jvm-4.0.2.jar
mysql-connector-java-6.0.6.jar
netty-buffer-4.1.31.Final.jar
netty-codec-4.1.31.Final.jar
netty-codec-dns-4.1.30.Final.jar
netty-codec-http-4.1.31.Final.jar
netty-codec-http2-4.1.31.Final.jar
netty-codec-socks-4.1.30.Final.jar
netty-common-4.1.31.Final.jar
netty-handler-4.1.31.Final.jar
netty-handler-proxy-4.1.30.Final.jar
netty-nio-client-2.1.0.jar
netty-reactive-streams-2.0.0.jar
netty-reactive-streams-http-2.0.0.jar
netty-resolver-4.1.31.Final.jar
netty-resolver-dns-4.1.30.Final.jar
netty-transport-4.1.31.Final.jar
netty-transport-native-epoll-4.1.31.Final-linux-x86_64.jar
netty-transport-native-unix-common-4.1.31.Final.jar
profiles-2.1.0.jar
protobuf-java-2.6.1.jar
protocol-core-2.1.0.jar
reactive-streams-1.0.2.jar
regions-2.1.0.jar
rxjava-2.1.14.jar
sdk-core-2.1.0.jar
slf4j-api-1.7.25.jar
snakeyaml-1.23.jar
sqs-2.1.0.jar
my-app.jar
tg-core-2.11-SNAPSHOT.jar
utils-2.1.0.jar
validation-api-1.1.0.Final.jar
vertx-auth-common-3.6.2.jar
vertx-bridge-common-3.6.2.jar
vertx-config-3.6.2.jar
vertx-config-hocon-3.6.2.jar
vertx-core-3.6.2.jar
vertx-dropwizard-metrics-3.6.2.jar
vertx-lang-kotlin-3.6.2.jar
vertx-lang-kotlin-coroutines-3.6.2.jar
vertx-web-3.6.2.jar
vertx-web-client-3.6.2.jar
vertx-web-common-3.6.2.jar
oripwk commented 5 years ago

Hi,

The problem is only solved when forcing dynamodb, kinesis, and cloudwatch to version 2.3.2.

This is not optimal because amazon-kinesis-client is at version 2.0.5 so there might be incompatibilities at runtime.

If you can find a more sustainable solution it would be great.

Thanks

zoewangg commented 5 years ago

Can you try to upgrade amazon-kinesis-client to 2.1.0? It should be compatible with the 2.3.2 sdk versions

clocklear commented 5 years ago

I was facing a similar issue (Class software.amazon.awssdk.core.protocol.StructuredPojo not found - continuing with a stub.) and changing my amazon-kinesis-client version to 2.1.0 resolved the issue.

Thanks!

debora-ito commented 5 years ago

@oripwk Are you still seeing the issue after @zoewangg 's suggestion?

oripwk commented 5 years ago

No. The issue was resolved by applying the suggestion. Sorry for forgetting to update.

debora-ito commented 5 years ago

Great! Closing this then, feel free to reopen if you have further questions.