bpot / poseidon

A client for Kafka 0.8
MIT License
260 stars 101 forks source link

Sporadic NoMethodErrors when calling Poseidon::PartitionConsumer.consumer_for_partition #98

Open edebill opened 8 years ago

edebill commented 8 years ago

I'm using Poseidon::PartitionConsumer.consumer_for_partition to get a PartitionConsumer on a cluster of kafka servers. Every once in a while it will blow up with a stack trace like this:

NoMethodError: undefined method `partition_leader' for nil:NilClass
  from poseidon/partition_consumer.rb:26:in `consumer_for_partition'
  from poseidon/broker_pool.rb:12:in `open'
  from poseidon/partition_consumer.rb:30:in `block in consumer_for_partition'
  from poseidon/cluster_metadata.rb:52:in `lead_broker_for_partition'

My calling code is like so:

  def consumer
    @consumer ||= Poseidon::PartitionConsumer.consumer_for_partition(@app, kafka_servers,
                                                                     @topic, @partition, @offset)
  end

  def kafka_servers
   ["host1:9092", "host2:9092", "host3:9092"]
  end

I'm able to rescue and retry (at which point it recovers), but it seems like a bad thing. What am I doing wrong?