Closed Suhail closed 14 years ago
As far as I understand reading None is not an issue at twitter. AFAIK: they have some (3, I guess) kestrel servers and a lot more workers. Each worker does the following: choose random server, read message, process message, repeat (may be chosen server is used for several iterations since operation like /close/open exists).
Anyway workers outnumber servers and choose server at random, so, eventually, each server would be read without any complicated logic.
I resolved it to the following:
When all queues are zero, it will simply just bounce between them (which is kind of shitty) so I set a higher /t=
both of those sound fine. we pick servers at random, and if the queue is empty, sleep for a while (exponential backoff) and then try another.
jkalucki has been experimenting with a much more aggressive client in the "grabby-hands" project.
Would it make sense to select a kestrel server based on the hash of a queue name? It seems to me that would be most efficient, though you may end up with small queues on some kestrel servers and large queues on others.
having a queue exist on only one (or even a subset) of the kestrels in a cluster would defeat the purpose of having the cluster, though.
It's been a year since this thread was active. Any news on ways to distribute kestrel? Preferred or best client policies?
we still generally have clients pick a server at random in ruby, using kestrel-client (https://github.com/twitter/kestrel-client). in java/scala, we have most clients connect to every server at once and launch a timeout-get on each one to minimize latency. those use grabby-hands (https://github.com/jkalucki/grabby-hands).
is that what you meant?
Yeah, that's exactly what I was wondering. I will look into those clients.
How exactly do you even distribute kestrel?
Most common memcached libs won't round robin read when it receives a None since they hash the key and horizontally partition.
What are people doing since I see nothing available short of building my own way to distribute tonight.