search

NEventStore / NEventStore.Persistence.MongoDB

Mongo Persistence Engine for NEventStore
MIT License
22 stars 26 forks source link

UpdateStreamHeadAsync upserts will error when using Mongo V3.0.3+ #35

Closed mikegore1000 closed 8 years ago

mikegore1000 commented 8 years ago

The Mongo query that the UpdateStreamHeadAsync method will fail on versions of Mongo V3.0.3+. This seems to be the case with both MMAP storage and WiredTiger.

BsonDocument streamHeadId = GetStreamHeadId(bucketId, streamId);
                    PersistedStreamHeads.Update(
                        Query.EQ("_id", streamHeadId),
                        Update
                            .Set(MongoStreamHeadFields.HeadRevision, streamRevision)
                            .Inc(MongoStreamHeadFields.SnapshotRevision, 0)
                            .Inc(MongoStreamHeadFields.Unsnapshotted, eventsCount),
                        UpdateFlags.Upsert);

We've only tested on MMAP, but the JIRA ticket states this is a WiredTiger issue too: https://jira.mongodb.org/browse/SERVER-14322

This only happens if two threads perform a commit at the same time, but as our app cannot catch the exception it terminates the process (raising a MongoDuplicateKeyException). In order to fix this a retry policy needs to be implemented so if the initial Upsert fails the second attempt then performs an update.

However, this does mean that the streams document isn't guaranteed to be the latest one, however this is true in the current implementation anyway due to the use of tasks to cover this.

The only other alternative I see is to swallow the exception and accept the failure here, which doesn't feel great.

alkampfergit commented 8 years ago

Actually it should be caused by the old legacy driver. The released version of MongoDb Adapter uses the 1.10 version of the driver, and it is not fully compatible with the new version of mongo.

In develop branch of MongoDb adapter we migrated to new driver, but it is for the next version of NEventStore, probably it would be a good idea to publish another version of MongDb persistence package that uses new driver version, to allow people to use new version of mongo.

mikegore1000 commented 8 years ago

According to https://docs.mongodb.com/ecosystem/drivers/csharp/ the 1.10.0 driver is compatible with Mongo 3.0, just not 3.1 onwards.

Can you let me know what feature in the driver would cause the upsert failure issue? Our service worked fine apart from this particular upsert issue which caused our processes to terminate.

We had less loaded services that ran with no issues at all.

mikegore1000 commented 8 years ago

I do agree though that pushing a new package to get the latest drive support would be a very good thing. Quite happy to help with a pull request on this.

I do think we'd still see the upsert issue though as it seems to be server related rather than client related.

alkampfergit commented 8 years ago

Ok, thanks a lot. Tomorrowwe will review the pull request.

mikegore1000 commented 8 years ago

Is the new version of the package going to be published to NuGet? I can't see it there at present.

alkampfergit commented 8 years ago

Sorry for the delay actually it is published on myget feed

https://www.myget.org/feed/neventstore-ci/package/nuget/NEventStore.Persistence.MongoDB

version is 5.3.5.

Could you please test it? If everything is ok we will push to Nuget feed.

mikegore1000 commented 8 years ago

Sorry it's taken so long to get back to you on this. We have just deployed the fix to production with Mongo V3 + WiredTiger and it's working great.

alkampfergit commented 8 years ago

Thanks a lor for confirming that the patch solved the issue.