search

cayasso / mongo-oplog

Watch mongodb oplog in a simple way
381 stars 91 forks source link

Timeout will restart cursor from same point as initial tail #19

Closed msamblanet closed 9 years ago

msamblanet commented 9 years ago

Right now, if a cursor timeout is received then tail() is called again. However, it will restart the tail with the same since value as originally requested as opposed to resuming where it was left off, resulting in duplication of any transactions still in the oplog but already delivered to the application.

A similar thing happens if you want to stop() and then call tail() again later to resume. In my case, I want to do this if I get DB transactions in faster than I can process them, effectively using the oplog as a buffer against activity spikes.

I believe simply updating this.ts inside of ondata to be doc.ts.getHighBits() would provide for this.

If this makes sense and assuming I did not miss anything, I could likely provide a PR some time in the next couple weeks if it would help.

jdarling commented 9 years ago

This makes a lot of sense. Hopefully this library gets some attention, I've had to move away from it due to errors, speed, and this reconnect/reset thing.

cayasso commented 9 years ago

@msamblanet @jdarling mongo-oplog@1.0.0 is out with latest fixes. Thanks for reporting issues and for being patient