search

awslabs / amazon-kinesis-connector-flink

This is a fork of the Apache Flink Kinesis connector adding Enhanced Fanout support for Flink 1.8/1.11 on KDA.
Apache License 2.0
20 stars 11 forks source link

FLINK-24431 Copy Flink connector changes to release-1.0 #49

Closed rudikershaw closed 2 years ago

rudikershaw commented 2 years ago

Issue #: FLINK-24431

Description of changes: The EFO Kinesis connector will register and de-register stream consumers based on the configured registration strategy. When EAGER is used, the client (usually job manager) will register the consumer and then the task managers will de-register the consumer when job stops/fails. If the job is configured to restart on fail, then the consumer will not exist and the job will continuously fail over.

I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

rudikershaw commented 2 years ago

FYI - I had some issues with some test race conditions on release-1.0. It looks like the test in question has since been changed to be more reliable. I have copied down those changes (In the FlinkKinesisProducerTest) into this pull request.

dannycranmer commented 2 years ago

FYI - I had some issues with some test race conditions on release-1.0. It looks like the test in question has since been changed to be more reliable. I have copied down those changes (In the FlinkKinesisProducerTest) into this pull request.

Thanks!