codelibs / elasticsearch-dataformat

Excel/CSV/BulkJSON downloads on Elasticsearch.
Apache License 2.0
145 stars 36 forks source link

Heap Allocation Problem #38

Open anujtmr9 opened 7 years ago

anujtmr9 commented 7 years ago

I have installed the plugin but whenever i m trying to retrieve data size more than 20mb i m facing memory exception. How can i handle default allocation and customize it

marevol commented 7 years ago

limit request parameter is available(ex. limit=50%).

anujtmr9 commented 7 years ago

Sorry i need to retrieve more amount of data nearly 1gb some time so how i can increase memory allocation. This is my question

marevol commented 7 years ago

It depends on a heap size of elasticsearch. limit=50% means 50% of the heap size.

anujtmr9 commented 7 years ago

Thanks you. How to set limit will u please help. Any specific command like so ?

marevol commented 7 years ago

limit is a request parameter.

anujtmr9 commented 7 years ago

Thanks. So in http request only like http://myurl/..../limit=50% it will work ?

marevol commented 7 years ago

?limit=50%25

anujtmr9 commented 7 years ago

bgl-mongo-rs1:9200/data1/logs/_data?format=json&source={"query":{"bool":{"must":[{"match_all":{}}]}},"from":0,"size":100000}

Error is "Result window is too large, from + size must be less than or equal to: [10000] but was [100000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting." Please update the url how i can set limit here