How do you export a large result set?

codelibs / elasticsearch-dataformat

Excel/CSV/BulkJSON downloads on Elasticsearch.

Apache License 2.0

144 stars 36 forks source link

How do you export a large result set? #41

Open xermus opened 6 years ago

xermus commented 6 years ago

The dataformat is perfectly running fine for me.

The problem is only that I need to export the whole index (which contains hundreds of thousands of documents). I then get:

Result window is too large, from + size must be less than or equal to: [10000] but was [1000000]

Can you provide me a specific example on how I can export all documents of an index to a given file (json or csv) on the Linux command line. Thank you.

marevol commented 6 years ago

"file" request parameter stores an exported file in elasticsearch server without storing data into a response. ex. localhost:9200/{index}/_data?format=csv&file=/tmp/test.csv...

sewalita commented 6 years ago

I am looking to export all documents from the index. Using plugin v5.6.2 with ES 5.6.4, I get the following error when using the "file" parameter

"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/{index}/{type}/_data] contains unrecognized parameter: [file]"}]