search

elastic / elasticsearch-migration

This plugin will help you to check whether you can upgrade directly to the next major version of Elasticsearch, or whether you need to make changes to your data and cluster before doing so.
290 stars 32 forks source link

Warn users when reindexing .kibana index using reindex helper #74

Closed ppf2 closed 8 years ago

ppf2 commented 8 years ago

Scenario:

1) Running 4.0 or 4.1 version of Kibana with ES 1.x, which means that the .kibana index is created using ES 1.x (requiring a reindex before upgrading Kibana to 5).

2) Upgrade ES to 2.x and use the migration plugin's reindex helper to reindex the .kibana index on while on ES 2.x. The reindex completed successfully and generated a .kibana-2.4.0 index and created an alias .kibana that maps to that index.

3) However, if the user tries to use Kibana 4.2+, i.e. a Kibana 4 instance that is compatible with ES 2.x (eg. Kibana 4.6) during this time against the .kibana index alias created, all sections of Kibana will stop working and produce blank screens:

image

This is because Kibana 4 doesn't handle .kibana as an index alias vs. an actual index - this is fixed in Kibana 5 (not backport-ed to 4.x)

Note that this is a possible use case for it can take a while to reindex all the older indices while on ES 2.x, so during this time, users can expect to still have a Kibana 4 instance up and running while reindexing is happening (and before they can finally upgrade to ES 5).

4) So if the user simply goes down the index list on the reindex helper screen and click to reindex, they can easily trigger a reindex of the .kibana index and then realize that they no longer have a working Kibana 4 instance anymore - unless they workaround this by updating the kibana.yml file to point kibana_index: to the direct index name (.kibana-) generated by the reindex helper instead of using .kibana.

I discussed with @tbragin @spalger and we think it will be helpful to add a warning to the reindex helper for the .kibana index.

tbragin commented 8 years ago

Note that even if we backport the fix to follow aliases to Kibana 4.6.x, there are already versions of Kibana 4.6 out there that would experience this problem for user simply reindexing everything that is shown by migration assistant as having the older format. So some sort of warning about exercising caution before reindexing .kibana index (e.g. first reading Kibana upgrade docs errata - which we still have to flesh out, btw) would be helpful.

tbragin commented 8 years ago

Update: Backport of the fix that follows aliases in Kibana is merged to 4.6.2, but yet to be released https://github.com/elastic/kibana/pull/8663

Versions 4.6.0 and 4.6.1 will continue to not work with the .kibana index alias.

eskibars commented 8 years ago

I think rather than dealing with the Kibana+Elasticsearch compatibility-matrix-of-death once again in this plugin, we should just warn the user something along the lines of Warning: you need to reindex to go to 5.x, but reindexing .kibana can create problems while you continue to stay on Kibana 4.x. You will need to upgrade Kibana immediately to maintain operations or to change kibana.yml to point to what will be the newly created kibana-X.Y.Z index

clintongormley commented 8 years ago

I've added a popup warning when reindexing kibana explaining what the user needs to do to continue using Kibana 4.x, and allowing them to cancel the reindex.