sclasen / akka-kafka

185 stars 62 forks source link

Feature Request: option to return message metadata (e.g. msg key) to consumer #8

Closed jbweeks closed 10 years ago

jbweeks commented 10 years ago

Right now lines 210-211 in Actors.scala look like:

Lines 210-211 in Actors.scala: /* ok to process, and msg available */ case Event(Continue, outstanding) if hasNext() => val msg = msgIterator.next().message()

Our producers are encoding keys with messages, we'd like the consumers to be able to access, but right now, the akka-kafka framework doesn't pass the meta-data up, AFAIK.

The Kafka MessageAndMetadata[K, V] class appears to have a method: key() that appears to be what we need...

Perhaps another optional property passed into the Connector constructor to return a tuple of (msg, key) instead of just message?

Thanks!

-Jonathan

sclasen commented 10 years ago

Agree that this should be there. Perhaps it should just send the whole MessageAndMetadata[K, V] along?

sclasen commented 10 years ago

Or perhaps the connector can take a function (MessageAndMetadata[K,V]) => Any that defaults to _.message()

jbweeks commented 10 years ago

Either of those sound fine, assuming this use-case doesn't have an adverse performance impact on the typical case where folks don't need the metadata. Thanks for the prompt attention! :)

sclasen commented 10 years ago

@jbweeks hey let me know if this approach is working for you, and I will cut a 0.0.6 release if so

jbweeks commented 10 years ago

This seems to be working for me, Scott. Thanks

sclasen commented 10 years ago

cut 0.0.6, should be in maven central shortly