search

reiseburo / hermann

A gem providing cross-platform Kafka producer and consumer support
MIT License
71 stars 76 forks source link

Add support for passing partition_key in MRI #121

Closed jacobsandlund closed 9 years ago

jacobsandlund commented 9 years ago

This PR adds support for consistent partitioning to MRI. The code chooses a consistent partition based on the hash of the :partition_key (CRC32). In the case where :partition_key is nil or an empty string, then it chooses a random partition.

The routines to compute a consistent hash were already in librdkafka, though they weren't exposed in the rdkafka.h header file, so we copied rdcrc32.h. I believe a later release of librdkafka may expose the rd_kafka_msg_partitioner_consistent function, but I don't think there is a tag with it, yet.

rtyler commented 9 years ago

This has been released under 0.26.1.0

jacobsandlund commented 9 years ago

Sweet!