search

Meteor-Community-Packages / meteor-collection-hooks

Meteor Collection Hooks
https://atmospherejs.com/matb33/collection-hooks
MIT License
658 stars 90 forks source link

How to deal with replication lag? #257

Open pouya-eghbali opened 4 years ago

pouya-eghbali commented 4 years ago

My after.update hooks return the old document because of replication lag (meteor writes the update to primary, collection hooks reads the document from secondary before it gets a chance to synchronize with primary).

How to deal with this? I cannot simply add a timeout to my hooks, that won't solve the issue.

StorytellerCZ commented 4 years ago

Yeah, timeouts are not a solution. With a quick look I'm wondering if #256 might potentially fix the issue without having to create a workaround.

pouya-eghbali commented 4 years ago

I checked #256 , I doubt if it helps with this situation but can give it a try. I can test and report tonight.

pouya-eghbali commented 4 years ago

Ok I tested #256 just now and it does not solve the issue I'm facing. As a temporary solution I'll just add a timeout to my fork. I'll try to find a real solution. Any suggestions? Ideas? Does anyone know any way to fetch documents from primary?

StorytellerCZ commented 4 years ago

@zimme @sebakerckhof

sebakerckhof commented 4 years ago

Wouldn't this need to be solved by setting appropriate write/read concerns? Of course this comes with a performance cost...

pouya-eghbali commented 4 years ago

Yeah, that'll solve the issue. However, I'll need to define a global write concern (couldn't find a way to set write concern for insert/update in meteor).

I'm fine with the performance cost, in my case faster read is more important compared to faster write. But then if one of the secondaries die the write operation returns with an error. I'm not a mongo expert, that's what I understood from mongo docs, and a few questions on stackoverflow.

SimonSimCity commented 4 years ago

The related ticket in the meteor repository (https://github.com/meteor/meteor/issues/10443) might also be worth a look.

pouya-eghbali commented 4 years ago 
readPreference: 'primary',

Suggested in meteor/meteor#10443 might solve the issue, but the entire point of having secondaries, at least in my case, is to read from secondaries.

Anyways, thank you all for your suggestions, I ended up making this. Posting here for reference.

evolross commented 4 years ago

I think I just ran into this issue too. I updated my production app's MONGO_URL to use readPreference=nearest because I'm trying to add support for my US-based us-east-1 Galaxy app in Europe by deploying to the European Galaxy on eu-west-1 (see Meteor forum post about issues). So I added a read-only node on Atlas in eu-west-1 for my European users to read.

It looks like this stopped doc from being up-to-date in my after.update hooks. Which consequently prevented only some of my users from upgrading in production... yikes!

I'm seeing quite a few errors around the app because of this. Perhaps updating my hooks to get their values from modifier instead of doc where possible is one work-around.

One weird thing is after adding readPreference=nearest this breakage with doc started happening to US users as well. Before the DB URL update to my app last night to include readPreference=nearest doc was always fine. I thought adding readPreference=nearest would just make European-based users find that new read-only node I added in eu-west-1. But apparently it's making all users find their nearest node which for many is out of date due to replication. Is this what happens when adding readPreference=nearest?