search

rackerlabs / otter

Rackspace Auto Scale
http://www.rackspace.com/cloud/auto-scale/
Other
53 stars 27 forks source link

Convergence ZK children limit #1294

Open manishtomar opened 9 years ago

manishtomar commented 9 years ago

Do a performance test to figure out maximum number of children that can be fetched by txkazoo. This limit will be useful to know how many groups can otter converge at a time. One optimization is mentioned in #1288 but that should not impact this limit.

radix commented 9 years ago

I'm a little confused about exactly what you're talking about here:

Do a performance test to figure out maximum number of children that can be fetched by txkazoo.

Here it sounds like you're talking about GetChildren calls being problematic with too many nodes? There's no way to limit how many children to fetch at once, so the only way we could limit this is by not adding child nodes in the first place (or perhaps by partitioning the dirty-flags into multiple different directories)

This limit will be useful to know how many groups can otter converge at a time.

But this sounds like something other than GetChildren. There may be any other number of things that will bottleneck highly concurrent convergence, not just listing ZK children (in fact I'd guess that we'll hit Nova problems first, but we should indeed find out). I think it'll be interesting to find out if we should add some limiter to convergence parallelism.

manishtomar commented 9 years ago

Here it sounds like you're talking about GetChildren calls being problematic with too many nodes?

Yes. Assuming you meant znode in place of node above.

There's no way to limit how many children to fetch at once, so the only way we could limit this is by not adding child nodes in the first place (or perhaps by partitioning the dirty-flags into multiple different directories)

I wasn't suggesting alternatives. I just want us to be aware of a sane limit of number of children that can be fetched. We can definitely implement any of the above alternatives once we know the limit and that limit is not good enough.

But this sounds like something other than GetChildren.

Yes. I am sorry I should've worded it correctly. This just happens to be one of the things that will provide information about the scale of convergence.