search

akka / alpakka

Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
https://doc.akka.io/docs/alpakka/current/
Other
1.26k stars 647 forks source link

alpakka kinesis is not consuming messages from localstack kinesis #2576

Open jbward1 opened 3 years ago

jbward1 commented 3 years ago

Versions used

val AkkaStreamAlpakka = "2.0.2" val AmazonKinesisClient = "2.3.3" val AWSSDK = "2.15.82"

Expected Behavior

After starting localstack kinesis, I ran a source to a Sink.head, then put a message in localstack kinesis.

 KinesisSchedulerSource(
      builder("testing-stream", kinesisClients),
      KinesisSchedulerSourceSettings.defaults
    ).runWith(Sink.head)

I expected the message I put in the kinesis stream to be returned

Actual Behavior

The Kinesis message doesn't arrive at the Sink.

Relevant logs

I have DEBUG logs turned on, but i'm not able to figure out where the issue is.

Reproducible Test Case

Please provide a PR with a failing test.

Alpakka kinesis consumer testing project: https://github.com/jbward1/alpakka-kinesis-consumer

seglo commented 3 years ago

Thanks for the issue @jbward1. We use the localstack Docker image in our build for integration tests for the aws eventbridge connector. It would be great if we could do something similar for the kinesis connector as well. Since you've spent some time working on this already I was wondering if you would be open to adding some integration tests to our build? Even a simple test would be sufficient that can be expanded on later.

seglo commented 3 years ago

Maybe you could investigate this in #2575, or a separate PR?