search

cloudant-labs / clouseau

Expose Lucene features as an erlang-like node
Apache License 2.0
57 stars 32 forks source link

Snapshots #48

Closed rnewson closed 2 years ago

rnewson commented 2 years ago

Add ability to create a snapshot of an index in a separate directory (using hard links, so must be same filesystem).

tonysun83 commented 2 years ago

Just had one more question regarding snapshot removal (see above thread).

Out of scope for this PR, but I'm curious whether this new functionality will be consumed with Dreyfus. From a couchdb perspective, could these snapshots also help with large indexing rebuilds? In other words, instead of having to rebuild everything for whatever reason, we can just start at the snapshot?

rnewson commented 2 years ago

Hi, Yes indeed. The purpose here is to allow administrators the ability to speed up database recovery. While restoring copies of the .couch files is sufficient to get you back and running it does mean waiting for all the indexes. The snapshots here will include the embedded CommitData which includes the update sequence we've reached, so clouseau will just index from that point and not zero.

As for dreyfus, I'm not sure if the api goes there as the motivation here is to add this ability for IBM/Cloudant usage in the first instance. Putting that in dreyfus means committing to a method that might not suit everyone and might change in an incompatible way in the near future.

tonysun83 commented 2 years ago

+1