apache / openwhisk-package-kafka

Apache OpenWhisk package for communicating with Kafka or Message Hub
https://openwhisk.apache.org/
Apache License 2.0
33 stars 43 forks source link

Set consumer state to running after brokers and partitions are assigned #340

Closed dubee closed 5 years ago

dubee commented 5 years ago

Currently the provider sets the consumer state to running when polling begins. However, this is not an accurate state as the call to subscribe a consumer is non-blocking. Meaning polling can begin before a consumer is connected to any brokers or assigned partitions, resulting in the consumer being set to a running state even though it is not possible to consumer messages yet. This can be problematic for integration tests that check the consumer health endpoint to see if a consumer is in a proper running state prior to producing test messages. To rectify this problem, the changes here set the consumer state to running only after brokers and partitions are assigned to the consumer.

dubee commented 5 years ago

To further explain, there may be interference by an unrelated consumer hanging around that is connected to the topic being used to test with, the actual test consumer may never receive messages produced by the test as the other consumer that is hanging around will consume the message before the test consumer is even connected to brokers and partitions. Waiting for the test consumer to connect to brokers and be assigned partitions before settings its state to running will result in both consumers receiving the message produced by a test eliminating the interference by unrelated consumers.