[bug] Missing messages when running in clusters

robertsLando commented 4 years ago

See https://github.com/moscajs/aedes-tests/pull/5#issuecomment-616449835

mcollina commented 4 years ago

That'd be interesting to fix! Have you got any ideas?

robertsLando commented 4 years ago

I'm investigating on this... I can't find any docs in mongodb about tailable capped collections in clusters with mentions about data loss. Also the problem could be here: https://github.com/mcollina/mqemitter-mongodb/blob/master/mqemitter-mongodb.js#L190

Do you have any ideas @mcollina ?

mcollina commented 4 years ago

I don't have time to look into this :/

robertsLando commented 4 years ago

Based on a question that I opened on mongodb forum:

https://developer.mongodb.com/community/forums/t/mongodb-pub-sub-using-tailable-cursors-in-nodejs-with-clusters/3027/2

Seems that talable cursors are not so good with clusters, I will submit a fix based on this suggestion but that will require mongodb >= 3.6. So what do you suggest? Check mongodb version and use this method if possible and keep the old one as a fallback?

Daniel - Software Engineer

Support me at: Github sponsors

On 21 Apr 2020, at 01:41, Matteo Collina notifications@github.com wrote:

I don't have time to look into this :/

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

mcollina commented 4 years ago

Ok, we'll bump a major!

robertsLando commented 4 years ago

I have found a fix, with this everything works with clusters and there is no need of major updates, just a patch :smile:

mcollina / mqemitter-mongodb

[bug] Missing messages when running in clusters #17

Seems that talable cursors are not so good with clusters, I will submit a fix based on this suggestion but that will require mongodb >= 3.6. So what do you suggest? Check mongodb version and use this method if possible and keep the old one as a fallback?