richardwilly98 / elasticsearch-river-mongodb

MongoDB River Plugin for ElasticSearch
1.12k stars 215 forks source link

River bug for mongo sharding environments results in STALE #509

Open aleph2 opened 9 years ago

aleph2 commented 9 years ago

We found the last timestamp value is keyed by namespace(db.collection), however for mongo, one collection can be stored in multiple shards. In this situation, last timestamp may from shard A, and the latest timestamp may from shard B. When above two timestamp mismatch, the STALE happened.