search

cayasso / mongo-oplog

Watch mongodb oplog in a simple way
381 stars 91 forks source link

Cursor is closed. #29

Closed gothammm closed 7 years ago

gothammm commented 8 years ago

I upgraded to v1, and since then I constantly get this error, on 'error' event.

{ [MongoError: Cursor is closed] name: 'MongoError', message: 'Cursor is closed', driver: true }

I'm not sure what's going on.

drgmr commented 8 years ago

Since it's been quite some time since this issue was open, I'd like to ask: @peek4y, did you solve your issue? I'm having trouble with it and no luck with google.

cayasso commented 8 years ago

Sorry I been bombarded with work. Is the cursor not reconnecting at all?

drgmr commented 8 years ago

I'm still looking into it, but it seems that on my dev machine the issue never happens, but in production (AWS) it occurs many times, missing a lot of events. Once or twice it ran for hours without reconnecting.

gothammm commented 8 years ago

Not really. We reverted back to the previous version. Dint check after that.

VigneshPeriasami commented 8 years ago

Currently I'm calling tail again on stream end for reconnection.

It is probably because of the outdated cursor tailing method on mongo-oplog-cursor. In latest node mongo driver documentation it uses the addCursorFlag method for tailing.

drgmr commented 8 years ago

I ended up implementing a cursor and "event streaming" system from scratch with great success, it's been running for more than two weeks now and not a single event was lost or any connection was closed. I tried closing the stream and tailing again on any error event and it ended up creating 400+ connections - that made me give up and create a solution using the latest mongo driver.

ebourmalo commented 8 years ago

it seems we've encountered the same issue. Nice job @edupc, any chance to see it published on a gist please?

drgmr commented 8 years ago

@ebourmalo here it is. The exported function expects something like this:

{
  collection: 'name',
  handlers: [
    {
      event: 'insert',
      handler: (oplogEntry) => {}
    },
    ...
  ]
}
cayasso commented 7 years ago

Hi sorry for my awful late response I know its been long time since this issue was opened. I just had a time to release 2.0.0 this release should address the issue in particular addressing a parent connection that was being opened but not closed https://github.com/cayasso/mongo-oplog/blob/1.x/index.js#L79

Feel free to open this issue if found any related issue. Thanks!