elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.73k stars 24.68k forks source link

Support CSV in `_bulk` #110314

Open xeraa opened 3 months ago

xeraa commented 3 months ago

Description

Importing CSV programmatically (without the use of Kibana) is surprisingly complicated; and often requires a fallback to miller, awk, or similar.

A feature that could support curl -H "Content-Type: text/csv" -XPOST "localhost:9200/myindex/_bulk" --data-binary @- would be very useful. And it could be limited in scope by design: only target a single index (myindex in this example) and require the first row to define the field names. But it should be possible to target an ingest pipeline like any other _bulk request to make changes to the structure as needed.

For reference, this is quite a common feature in other datastores:

elasticsearchmachine commented 3 months ago

Pinging @elastic/es-distributed (Team:Distributed)