search

elastic / stream2es

Stream data into ES (Wikipedia, Twitter, stdin, or other ESes)
355 stars 62 forks source link

consider document version in es -> es streaming #25

Closed jimabramson closed 9 years ago

jimabramson commented 10 years ago

since ES index APIs support optimistic concurrency control using document versioning, it would be very useful to support this functionality when using stream2es to migrate indexes.

In particular, it would allow for idempotency when copying data incrementally (one could copy documents from an older index's data conditionally, only applying updates or deletes to the new index when no subsequent change had been already made to those documents there).

drewr commented 9 years ago

Whoops! Long time no reply. :disappointed:

@jimabramson The problem here is that _version is an ephemeral value. It's doesn't keep incrementing forever, and the values wouldn't be meaningful cross-index.