bitemyapp / bloodhound

Haskell Elasticsearch client and query DSL
bitemyapp.com
BSD 3-Clause "New" or "Revised" License
424 stars 118 forks source link

Snapshot restore #126

Closed MichaelXavier closed 8 years ago

MichaelXavier commented 8 years ago

Here it is! I have this running in production taking daily snapshots.

One immediate change I want to make that I'll do in another commit is that the Client module sorely needs haddock headers so we can break up the ever growing array of client functions by subject.

One thing you may have an opinion on is the decision to use a typeclass (which is not lawless but the law is nothing fancy) to represent snapshot repos. The idea there is that there will be mini packages for all of the pluggable snapshot repo types that es supports, e.g. S3 backups using amazonka types. I actually have one vendored in my app that I plan on breaking out if this gets through.

MichaelXavier commented 8 years ago

Oh yeah, tagging in #124

MichaelXavier commented 8 years ago

So it looks like there's some variability between ES versions supported in the test suite. I think I'll add a function to get the elasticsearch info object that includes the version and then configure the tests accordingly. It would be a shame to not test some of the newer features, as would dropping old versions of es from the test suite over a small part of a few tests. I'll also document in the types which features appeared in which version.

MichaelXavier commented 8 years ago

@bitemyapp ok we're good now. I documented some of the differences for different ES versions and updated the tests to make a best effort to run the snapshot tests. Note that for development if you use an ES 1.6 or newer and don't have a repo set up in your options (which is the default case), snapshot tests will have to be skipped. CI now specifies the repo when installing elasticsearch so each cell in the build matrix runs as many of the tests as possible.

MichaelXavier commented 8 years ago

@bitemyapp Wanted to gently prod you on this to see if you had any reservations about merging this. We've been running it in production to manage daily snapshots for the last week or so without issue.

bitemyapp commented 8 years ago

Yes it's good to go, sorry for the delay!