wxf4150 / esdump

elasticserch dump import export
8 stars 5 forks source link

export which match body

./esdump export --es http://server1:9200 --MatchBody '{"range": {"eventTimestamp": {"gte": "2021-05-07T10:32:20.170178Z"}}}' --index events

export data a minites ago

./esdump export --es http://server1:9200 -m "{\"range\": {\"eventTimestamp\": {\"gte\": \"date -d "1 minutes ago" +%s000\"}}}" --index events -o - | ./esdump import --index events -i - ./esdump -h ./esdump import -h ./esdump expport -h


**note**:
- when use import;  you should setting the target index's _mapping .

command help:
```shell script
 ./esdump -h
es import export

Usage:
  esdump [flags]
  esdump [command]

Available Commands:
  export      elasticsearch export
  help        Help about any command
  import      elasticsearch import

Flags:
      --es string      es url (default "http://localhost:9200")
  -h, --help           help for esdump
      --index string   index name (default "my_index")

Use "esdump [command] --help" for more information about a command.

./esdump  export -h
elasticsearch export

Usage:
  esdump export [flags]

Flags:
  -m, --MatchBody string   MatchBody, empty for match_all; example:{"range": {"timestamp": {"gte": "2021-04-20"}}} (default "{\"match_all\":{}}")
  -h, --help       help for export
      --o string   export desk filename; use - for stdout (default "./tmp_export.json.gz")

Global Flags:
      --es string      es url (default "http://localhost:9200")
      --index string   index name (default "my_index")

 ./esdump import -h
elasticsearch import

Usage:
  esdump import [flags]

Flags:
  -h, --help       help for import
      --i string   import  filename; use - for stdin (default "./tmp_import.json.gz")

Global Flags:
      --es string      es url (default "http://localhost:9200")
      --index string   index name (default "my_index")

why it so quick?

note: res.hits.source is the document body from elasticsearch respose body

the export format is below and very simple:

{"ID":"163820696","RawData":{"id":163820696,"asset":"","imageUrl":""}}
{"ID":"163820697","RawData":{"id":163820696,"asset":"","imageUrl":""}}
{"ID":"163820698","RawData":{"id":163820696,"asset":"","imageUrl":""}}
...
...

one document one row. the field "RawData" in the document.

sorry my bad english