search

awslabs / dynamodb-streams-kinesis-adapter

The Amazon DynamoDB Streams Adapter implements the Amazon Kinesis interface so that your application can use KCL to consume and process data from a DynamoDB stream.
Apache License 2.0
97 stars 37 forks source link

wrong aws java sdk version specified in pom.xml #19

Closed zerth closed 6 years ago

zerth commented 6 years ago

pom.xml currently specifies version 1.11.218 for various aws sdks: https://github.com/awslabs/dynamodb-streams-kinesis-adapter/blob/a90496512a87852bd61fce8791fd53acc079ed7a/pom.xml#L26

However, it also specifies use of 1.9.0 for the KCL, which makes use of kinesis.model.ListShardsRequest, which appears to have been added in 1.11.272. As a result, this error occurs in programs using dynamodb-streams-kinesis-adapter 1.4.0 if they only install the dynamo streams adapter and its dependencies without separately installing KCL 1.9.0:

SEVERE: Caught throwable while processing data.
java.lang.NoClassDefFoundError: com/amazonaws/services/kinesis/model/ListShardsRequest
    at com.amazonaws.services.kinesis.clientlibrary.proxies.KinesisProxy.listShards(KinesisProxy.java:291)
    at com.amazonaws.services.kinesis.clientlibrary.proxies.KinesisProxy.getShardList(KinesisProxy.java:365)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ShardSyncer.getShardList(ShardSyncer.java:319)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ShardSyncer.syncShardLeases(ShardSyncer.java:121)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ShardSyncer.checkAndCreateLeasesForNewShards(ShardSyncer.java:90)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.ShardSyncTask.call(ShardSyncTask.java:71)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.MetricsCollectingTaskDecorator.call(MetricsCollectingTaskDecorator.java:49)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker.initialize(Worker.java:504)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker.run(Worker.java:436)
    at com.amazonaws.services.kinesis.multilang.MultiLangDaemon.call(MultiLangDaemon.java:114)
    at com.amazonaws.services.kinesis.multilang.MultiLangDaemon.call(MultiLangDaemon.java:61)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.amazonaws.services.kinesis.model.ListShardsRequest
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 15 more
zerth commented 6 years ago

Closing because this is really the responsibility of the using application / client packaging scheme. Using the released dynamodb streams adapter package works when used as intended for processing dynamodb streams.