search

orbitdb / field-manual

The Offical User's Guide to OrbitDB
208 stars 43 forks source link

FAQ: Is every 'put' to OrbitDB immediately sent to the network and persisted? #66

Open haadcode opened 5 years ago

haadcode commented 5 years ago

Is every 'put' to OrbitDB immediately sent to the IPFS network and when 'await orbitdb.put(key, value)' returns, can I assume that the data has been persisted to atleast one other node in addition to my node ?

When calling put or any other update operation on a database, the data is 1) saved locally and persisted to IPFS and 2) send to the network, through IPFS Pubsub, to peers who have the database open (ie. peers).

Upon calling put (or other updates), OrbitDB saves the data locally and returns. That is, the operation and its data is saved to the local node only after which put returns and asynchronously sends a message to pubsub peers. OrbitDB doesn't have a notion of confirming replication status from other peers (although this can be added on user-level) and considers operation a success upon persisting it locally. OrbitDB doesn't use consensus nor does it wait for the network to confirm operations making it an eventually consistent system.

In short: it can't be assumed that data has been replicated to the network after an update-operation call finishes (eg. put, add).

haadcode commented 5 years ago

Related to: orbitdb/orbit-db#342, orbitdb/orbit-db#474, orbitdb/orbit-db#212

tyleryasaka commented 5 years ago

I'm finding it challenging to easily determine when my database has finished syncing with all connected peers (i.e. when the last replicated event is fired). Would be useful for me if there was some sort of replicationFinished event (or something like that) which would let me know that I don't need to wait for any more updates to my database. I realize there's no way to know if a database is "absolutely" in sync, but I'd just like to know when all of the in-progress operations have finished. Any chance something like this could be added? Or is there a better approach?

Edit: this would solve the problem described in https://github.com/orbitdb/orbit-db/issues/474. The first get fails because await db.load() resolves before the replication has finished. By the time the code is called a second time, the database has finished replicating. If instead of waiting on await db.load(), there was another event to wait on (maybe await db.sync()?), then db.get('test') would return the expected value on the first attempt.

aphelionz commented 4 years ago

Moving this to the Field Manual repo