taraslayshchuk / es2csv

Export from an Elasticsearch into a CSV file
Apache License 2.0
510 stars 191 forks source link

2 issues, doesn't respect "size" in query, and added extra 'CRLF's in my CVS file #23

Closed teebu closed 7 years ago

teebu commented 7 years ago

Query had "size" param, this program ignored it. I had to use -m.

When the CVS was generated, it has extra enters after each doc entry.

image

I should also include I am using this in Windows in Git Bash

taraslayshchuk commented 7 years ago

Hello

Looks like as Windows specific issue with end line symbol. As I don't have windows box I can not test it. Could you please provide example query with "size"?

teebu commented 7 years ago
{
  "_source": { "include": ["@timestamp","agent","cs_uri_stem","uri_query","clientip"]},
  "size": 10,
  "query": {
    "bool": {
      "minimum_should_match": 1,
      "must": [],
      "must_not": [],
      "should": []
    }
  }  
}
taraslayshchuk commented 7 years ago

With scroll api the size parameter is used in another case:

The size parameter allows you to configure the maximum number of hits to be returned with each batch of results. Each call to the scroll API returns the next batch of results until there are no more results left to return, i.e. the hits array is empty.

So you should use --max 10 instead of size attribute. By the way, "_source": { "include": ["@timestamp","agent","cs_uri_stem","uri_query","clientip"]} can be replaces with -f @timestamp agent cs_uri_stem uri_query clientip

teebu commented 7 years ago

thanks