crate / elasticsearch-inout-plugin

An Elasticsearch plugin which provides the ability to export data by query on server side.
Apache License 2.0
112 stars 15 forks source link

Way of managing kindness of _export or _dump #34

Open gmaurice opened 11 years ago

gmaurice commented 11 years ago

One time I launched a _dump on all the indexes available on my elasticsearch cluster, the node suffered of an overloading. The node, in normal working, has a system load about 1. When the _dump was launched, system loads has grown up to 22. I think _dump on several indexes tries to dump shards as many as possible.

I think doing the _dump one index a time is a better way to do.

However, what happens if you have many shards for an index ?

Is it possible to manage a sort of kindness inside plugin ? (Max dumping shards at a time, ...)