search

bpot / poseidon

A client for Kafka 0.8
MIT License
260 stars 101 forks source link

Is there no support for consumer groups? #94

Open messutied opened 9 years ago

messutied commented 9 years ago

I'm sorry this isn't really an issue but I wasn't sure where to ask.

When creating a consumer instance the format is 

consumer = Poseidon::PartitionConsumer.new(client_id, host, port, topic, partition)

Do I understand correctly that client_id is nothing related to a consumer group, and that poseidon does not currently support consumer groups?

If so, whats the purpose of the client_id? I see its used in the connection but I can't find references on the kafka documents to it.

Thanks

ThomasAlxDmy commented 9 years ago

+1 @bpot

michaeldauria commented 9 years ago

That is correct, an implementation of consumer groups is here: https://github.com/bsm/poseidon_cluster

It only covers some of the cases required for running a consumer, so make sure you test failure scenarios. A problem we have with it is that it does not handle NotLeaderForPartition errors and we have to restart consumers to fix it.