search

shikhar / kafka-connect-dynamodb

Kafka Connector for DynamoDB
Apache License 2.0
34 stars 50 forks source link

Dynamo Db source issue #19

Open Dev20152015 opened 6 years ago

Dev20152015 commented 6 years ago

Connector keeps on getting this issue: org.apache.kafka.connect.errors.ConnectException: No remaining source shards at dynamok.source.DynamoDbSourceTask.poll(DynamoDbSourceTask.java:84) at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:186) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:170) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:214) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

anupkumarsharma commented 6 years ago

Can you share your properties files? Shard is retrieved by AmazonDynamoDBStreamsClient. You also might want to check if the streaming is enabled and all the configurations of AWS are correct.

Dev20152015 commented 6 years ago

I figured out the issue. It was an AWS permission. Supposedly having admin access on the account running connector is not enough. You have to provide explicitly dynamodb access.

ravi72munde commented 5 years ago

@Dev20152015 Can you please elaborate what you did to solve this?

anupkumarsharma commented 5 years ago

@Dev20152015 - It would be helpful if you can provide more details on your resolution.