search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.71k stars 24.67k forks source link

Reindex from remote cluster: teach elasticsearch how to create the target index properly #52569

Open minfrin opened 4 years ago

minfrin commented 4 years ago

As described at https://www.elastic.co/guide/en/elasticsearch/reference/current/reindex-upgrade-remote.html, elasticsearch can copy data from one cluster to another.

Unfortunately, elasticsearch cannot copy an index from one cluster to another, a limitation that documented here:

 "2a: Create an index the appropriate mappings and settings."

In our case the entity that creates the index is a large commercial application that is not under our control, and "create an index the appropriate mappings and settings" messes up our architecture.

Ideally the index should be created on the target and the settings of the source index - where they make sense - be used to create the target index.

Right now today, the index is created on the target with default settings. After the copy is complete, the large commercial application is broken, as the index is incorrectly configured.

elasticmachine commented 4 years ago

Pinging @elastic/es-distributed (:Distributed/CRUD)

ppf2 commented 3 years ago

+1 This is a very common miss in the field when using the reindex API. While we document this limitation and ask the users to manually prepare the target index before running reindexing, it is a better experience if we can copy settings/mappings to the target index as part of reindexing (or make this behavior configurable).