Document write conflict resolution

[danielmewes]

jlu on Hacker News asked:

Does it handle conflict resolution, ike when two people modified the same field at almost the same time?

I know that we use versions internally to detect concurrent modifications between reading a document and a call to replace.

I'm not sure if this also applies to upsert. I'm also not sure what error we emit on the write operation if there is a conflict. @Tryneus can you clarify?

We need to document this and provide some guidance for how to deal with conflicts. I'm not sure where this would go best. Maybe just into the Collection API entries for the write operations? Or into the FAQ, and then we can link it from the API docs?

In general I think we should also provide error handlers for write operations more consistently throughout the example codes to illustrate that writes can fail (by using subscribe on the write terms).

[Tryneus]

Because validators are run on the horizon server, and not on the database, if two clients attempt to write to the same row at the same time, one will have a conflict and fail. This applies to almost all combinations of writes between the two clients - the only case I can think of off the top of my head that wouldn't error is if both clients are attempting to remove the document.

There is no reason this needs to happen when the requests match rules without validators, as then we could resolve the conflict atomically in the database. This would be solved by rethinkdb/horizon#380, which is not a huge task, but I wanted to keep the testing surface area small right before release.

As for individual writes and their error cases, they are as follows:

• All writes may fail if they do not pass the template and optional validator of at least one rule available to the user account of the current session
• insert
  • Fails if the document with that ID already exists (if no ID is specified, insert should never fail)
• remove
  • Fails if the document version has changed between when we fetch it for the validator and when we perform the remove
• replace
  • Fails if the document is not present
  • Fails if the document version has changed between when we fetch it for the validator and when we perform the replace
  • Fails if the document was removed between when we fetch it for the validator and when we perform the replace
• update
  • Fails if the document is not present
  • Fails if the document version has changed between when we fetch it for the validator and when we perform the update
  • Fails if the document was removed between when we fetch it for the validator and when we perform the update
• store (insert with replace)
  • Fails if the document version has changed between when we fetch it for the validator and when we perform the store
  • Fails if the document was removed between when we fetch it for the validator and when we perform the store
  • Fails if a document with the same ID was created between when we fetch it for the validator and when we perform the store (if no ID is specified, store should never fail)
• upsert (insert with update)
  • Fails if the document version has changed between when we fetch it for the validator and when we perform the upsert
  • Fails if the document was removed between when we fetch it for the validator and when we perform the upsert
  • Fails if a document with the same ID was created between when we fetch it for the validator and when we perform the upsert (if no ID is specified, upsert should never fail)

[danielmewes]

Thanks @Tryneus .

I thought we were pushing the version through to the client to catch cases like

• client A fetches a document
• client A modifies it locally, while
• client B changes the document through a separate write
• client A writes back the modified document through replace

I thought the last step would fail because the versions don't match, but I guess we didn't end up actually implementing this?

[Tryneus]

@danielmewes, you're right, that is still possible to do, actually - although you have to do it explicitly in the client using the new version values you receive from your write responses.

If a client explicitly specifies the version that they want to modify, we use that version when performing checks rather than the version from the initial fetch for the validator.

[danielmewes]

Oh cool, but how do I specify the version to a write? Do I just place the version field that I get from a write response into the document for the next write?

Is there a way for getting the current version of a document through a read, without writing to it first?

[Tryneus]

There isn't a current way to get the version of the document through a read as far as I know. The server does send this information to the client, but the version field is filtered out by the client at the moment to avoid users accidentally performing stricter writes that may cause problems at high loads (two clients performing strict writes repeatedly will always favor the client with the lowest latency). Perhaps there should be an option in the client to stop this filtering.

[Tryneus]

And yes, if you want to perform a write that will only apply to a certain version, you just include the $hz_v$ version field in the document to be written.

[coodoo]

jlu on hackernews, just chiming in to say thanks for tracking this issue, good job!