yjs / y-protocols

Yjs encoding protocols
MIT License
109 stars 39 forks source link

Add a callback after an update is applied to a y-doc, and pass the update in the callback #11

Closed kapv89 closed 3 years ago

kapv89 commented 3 years ago

Isolating the update that is received by a horizontally-scalable websocket-server from a client is essential for:

  1. persisting updates to y-doc - as different clients will send different updates, and only the updates that are sent by clients need to be persisted for having the ability to initialize a y-doc for a new client from the database
  2. broadcasting the update using pub-sub - as different websocket processes will receive different updates from the clients connected to them, broadcasting the incoming update is essentially how different y-docs held in memory by different websocket-servers can be kept in sync (and thus, the different clients can be kept in sync)

Having the callback added in this PR basically makes 1 & 2 possible without having the need to repeat some parts of what y-js does internally in project-specific code in order to isolate the to-be-applied update

dmonad commented 3 years ago

Hi @kapv89 ,

Applying a document update to ydoc will trigger an update event if anything changed. You should always prefer the update event because another client might already have sent that update to you, and you want to ignore the change if it is already known.

Is there any reason why you can't use the update event?

kapv89 commented 3 years ago

Hi @dmonad ,

In a horizontally scalable websocket backend for yjs, the update on ydoc will trigger whenever ydoc is updated as a result of an update sent from a client and when ydoc is updated as a result of pubsub that happens b/w websocket-server-processes. The purpose of this PR is to isolate the update sent by clients to the websocket server so that the update can be published on a pubsub channel, and be persisted into the database.

I've uploaded a slightly reworked example of a horizontally-scalable yjs-ws-server with persistence to postgres over here https://github.com/kapv89/yjs-scalable-ws-backend (will add a readme this weekend). If you look at the code in this area https://github.com/kapv89/yjs-scalable-ws-backend/blob/main/src/setupWSConnection.ts#L109-L141 ... I have tried to help reader understand what will be achieved by this PR

dmonad commented 3 years ago

Maybe you can have a look at some of my other bindings. Here, for example I use tho transaction.origin to "filter" updates and don't send them again to the server. This is the idiomatic approach to tell where an update is created from.

kapv89 commented 3 years ago

@dmonad yes, transaction.origin is sufficient to do the job. closing this PR