search

Tokutek / mongo

TokuMX is a high-performance, concurrent, compressing, drop-in replacement engine for MongoDB | Issue tracker: https://tokutek.atlassian.net/browse/MX/ |
http://www.tokutek.com/products/tokumx-for-mongodb/
703 stars 97 forks source link

add an option to disable the splitVector check when using a hashed shard key #395

Open leifwalsh opened 11 years ago

leifwalsh commented 11 years ago

splitVector adds a cost to each insert/update operation, and this cost is significant for non-clustering indexes. With a hashed shard key, you really shouldn't need to split much if at all. We should maybe disable the splitVector check (or at least make it a lot less frequent) if we're using a hashed shard key. Probably only if the user says they want us to disable it.

trebogeer commented 11 years ago

Seems like it's already there - --noAutoSplit for mongos.

leifwalsh commented 11 years ago

Cool! Let me take a look at what that does and then I'll close the ticket. Have you tested that it resolves your problem?

Cheers, Leif

On Mon, Aug 5, 2013 at 6:23 PM, Dmitry Vasilyev notifications@github.com wrote:

Seems like it's already there - --noAutoSplit for mongos.

Reply to this email directly or view it on GitHub: https://github.com/Tokutek/mongo/issues/395#issuecomment-22145485

leifwalsh commented 11 years ago

noAutoSplit disables this check for all operations through the mongos on which it is set. If you want to do this on a per-collection basis, it's not so simple. One option is to route all (modification) traffic to the collection that's sharded with a hashed key through the mongos(es) that have noAutoSplit set. Another option is for us to implement a collection-level option to control this. This would probably need to be stored on the config server and propagated to all mongos nodes.

@trebogeer let me know if you'll be ok using noAutoSplit for now or if we should think more about what it would take to solve this for you.

trebogeer commented 11 years ago

noAutoSplit is ok for now. Though it does not solve the original issue with timeouts. I'll email you the stack trace.