Closed carminexx closed 8 years ago
It's recommended to use bufferproducer, this will manually select the kafka partition (just simple round). And bufferproducer means higher performance.
Thanks for your reply; but I noticed that the partition is automatically selected even in buffer producer. For "manually" I mean that I can choose exactly to which partition send the message.
Sorry for the misunderstood, but I not exactly understand your idea. I don't think It's a good idea to specifie a partition in send api. Do you mean specifie a key in send api, and a partitioner function in producer config. The partitioner choose partition according to the key?
I have look into some kafka client in other language(like go) and the kafka protocol, seems the second one is better. And I don't see much sense to provide this in openresty world, could you show me?
I mean something in order to have, for example, messages relatives to same user in the same Kafka partition. An example: I have a web application with user authentication and user tracking via cookies. I want that messages relatives to an user are sent to a specified partition, and the partition will be chosen via LUA after an analysis of a user tracking-code, stored in a HTTP cookie. Ex: user 1 => tracking code 2 => all messages sent to partition 1 user 2 => tracking code 3 => all messages sent to partition 2 user 3 => tracking code 8 => all messages sent to partition 1 (in this case, via LUA code, I sent messages for users that have an even tracking code to partition 1; whereas users that have an odd tracking code have messages sent to partition 2).
Thanks, I got you. I will implement a new send api in the two days, or maybe done tomorrow.
Sorry, later for the $ work, may be finished in this weekend.
Sorry for the later again. Finally I finished a implement in the master branch now. Any feedback welcome.
Consider it resolved.
Hi, Is it possibile to manually select the kafka partition, when sending messages? I noticed that in the current implementation all messages are sent to partition 1, is it correct?