search

elastic / stack-docs

Elastic Stack Documentation
Other
95 stars 244 forks source link

Improve language around backups before an upgrade #25

Open zuketo opened 6 years ago

zuketo commented 6 years ago

In the "Preparing for an upgrade section" of this page, we mention "Back up your data. You cannot roll back to an earlier version unless you have a backup of your data. For information about creating snapshots, see Snapshot and Restore."

Due to the incremental nature of snapshots, we may want to mention that a new repo must be created after upgrading indexes in 5.6 (and before the upgrade is performed). Otherwise, the user will need to perform a snapshot restore on a version of ES < 6.0.

One possibility for changing the language:

"Back up your data. You cannot roll back to an earlier version unless you have a backup of your data. Note: snapshots are incremental, snapshot repos created in earlier versions of Elasticsearch may not be restorable in newer versions of Elasticsearch. For this reason, it is important to upgrade all indexes in 5.6, create a new snapshot repo, perform a snapshot, and then upgrade Elasticsearch. For information about creating snapshots, see Snapshot and Restore."

CC @eskibars

zuketo commented 6 years ago

I'm still investigating whether my statement above regarding restoring snapshots to 6.x is completely correct.

Here is an additional note: "if in 5.x you got rid off all 2.x indices and all your internal indices were created in 5.x it is possible to upgrade to 6.x without upgrading the indices first. However, security is not going to work until you upgrade the .security index and watcher is not going to work until you upgrade the .watches index."

We'll also need to update the language on this page: https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-snapshots.html

“A snapshot of an index created in 2.x can be restored to 5.x. A snapshot of an index created in 1.x can be restored to 2.x. A snapshot of an index created in 1.x can not be restored to 5.x.”

eskibars commented 6 years ago

Strictly speaking, it's not a requirement to create a new repo / reindex every time you move major versions (I could have started off on 5.0, snapshotted through the major, and upgraded to 6.0 and we'd still be perfectly fine reading the old snapshot), but it is generally a best practice when possible. Also, you could create a snapshot repository in 5.0 that contains an index that was created in 2.x and that would be unrestorable once you move to 6.0. I think we should strive to convey a few warnings and best practices:

As an aside, our snapshot/restore is hugely valuable but the semantics are pretty challenging and we'll need to address that over time. Having to write this type of documentation is challenging at best and that's generally a sign of a difficult-to-use feature.

ppf2 commented 6 years ago

Thanks, these are good ones to add to the doc.

This is probably a slight digression, but since it's related, I will ask here :) Have we considered prioritizing these features in snapshot ( https://github.com/elastic/elasticsearch/issues/20867) so users can more easily tell from what version of ES the index files snapshot-ed were originally created, and also provide an option to skip the incompatible ones so they can at least restore the compatible ones. This will help with upgrade experience related to snapshots in the future.

Thanks, Pius

On Thu, Sep 14, 2017 at 3:10 PM, Shane Connelly notifications@github.com wrote:

Strictly speaking, it's not a requirement to create a new repo / reindex every time you move major versions (I could have started off on 5.0, snapshotted through the major, and upgraded to 6.0 and we'd still be perfectly fine reading the old snapshot), but it is generally a best practice when possible. Also, you could create a snapshot repository in 5.0 that contains an index that was created in 2.x and that would be unrestorable once you move to 6.0. I think we should strive to convey a few warnings and best practices:

  • Indices that were created in 2.x are unreadable in 6.x. I think we do a good job of conveying this already.
  • Snapshots are incremental and may contain multiple indices that were created on various versions of Elasticsearch. This means if any of the indices in the snapshot were created prior to 5.0 (the "initial" bit before the incremental bits on top of it), they'll need to be reindex into a new snapshot repository in order to be restorable in 6.x. I think a warning like this could go in both the snapshot/restore docs and in the stack upgrade docs.
  • It's best practice to create a new snapshot repository for each major version. I don't think we have this anywhere, and I'd be in favor of it being in the snapshot/restore docs.

As an aside, our snapshot/restore is hugely valuable but the semantics are pretty challenging and we'll need to address that over time. Having to write this type of documentation is challenging at best and that's generally a sign of a difficult-to-use feature.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/elastic/stack-docs/issues/25#issuecomment-329622884, or mute the thread https://github.com/notifications/unsubscribe-auth/AG4dCaRU-xP6x7e-aSGlWNX5N6W4G0ybks5siaRAgaJpZM4PU7Ya .

eskibars commented 6 years ago

We want to more or less completely revisit how snapshot/restore works.

ppf2 commented 6 years ago

Ah cool, got it. thx Shane!

On Thu, Sep 14, 2017 at 5:07 PM, Shane Connelly notifications@github.com wrote:

We want to more or less completely revisit how snapshot/restore works.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/elastic/stack-docs/issues/25#issuecomment-329641547, or mute the thread https://github.com/notifications/unsubscribe-auth/AG4dCZ7OWKccdeR6LmeHexgSb11fTNdIks5sib-zgaJpZM4PU7Ya .