search

etspaceman / kinesis-mock

A Kinesis Mock using Http4s
MIT License
56 stars 17 forks source link

SubscribeToShard support #9

Open etspaceman opened 3 years ago

etspaceman commented 3 years ago

This one is a challenge. It requires Http2 server-push functionality and the documentation on this is very limiting. I believe Http4s has support for server-push but perhaps not in the way that SubscribeToShard needs to be supported.

https://docs.aws.amazon.com/kinesis/latest/APIReference/API_SubscribeToShard.html https://aws.amazon.com/blogs/aws/kds-enhanced-fanout/

Without this support, users will have to use the PollingConsumer in their tests.

etspaceman commented 3 years ago

There isn't a single Scala http server today that supports the PUSH_PROMISE use-case in http2. I opened a couple of issues to track support for this in Http4s

http4s/http4s-servlet#8 https://github.com/http4s/http4s/issues/4625

charles-slc commented 1 year ago

Curious if there's been any movement on this. Off the top of my head, it wouldn't seem that using a polling consumer in testing and a subscription consumer in production results in a valid test. But this is my teams first Kinesis project, I need to go back and study the differences.

etspaceman commented 1 year ago

Hey @charles-wilt .

There is not movement on this unfortunately. The support provided by http4s today for http2 does not allow for the delivery mechanism we need.

However... if you use localstack, they support SubscribeToShard by proxying it through GetRecords. So you can do the local tests that you are referring to.