cityindex-attic / logsearch

[unmaintained] A development environment for ELK
Apache License 2.0
24 stars 8 forks source link

Replace EC2 image/snapshot based with Elasticsearch snapshot based backup #321

Closed sopel closed 9 years ago

sopel commented 10 years ago

This depends on #320 - Elasticsearch 1.0 features a dedicated backup/restore mechanism, see Introducing Snapshot & Restore:

With most/all instances being auto deployed and scaled anyway, the current catch all approach is more of a simple stop gap measure than a reasonable and scalable backup process, which should primarily be concerned with the data.

:information_source: on top of that it might also address related scaling and resilience issues:

We are very excited about this new feature. We like to think of incremental backup as a time machine for your data. We are confident that everyone who relies on Elasticsearch as a critical component in their system and cannot afford down time for re-indexing will find the new Snapshot and Restore mechanism really helpful. [emphasis mine]

This relates to e.g. #272/#273 and esp. #313 accordingly - @dpb587, @mrdavidlaing: I've been planning to summarize my thoughts around the implicit and explicit usage (or current lack thereof) of the Event Sourcing pattern within our cluster for over a week, but didn't get around to it yet - so in an extreme nutshell (there's much more to it, and as always a few issues as well):

sopel commented 9 years ago

Closed as Won't Fix due to project being retired to the CityIndex Attic.