automerge / automerge-classic

A JSON-like data structure (a CRDT) that can be modified concurrently by different users, and merged again automatically.
http://automerge.org/
MIT License
14.76k stars 467 forks source link

Nodes disconnected and loop when connecting a new one #455

Closed joseluratm closed 2 years ago

joseluratm commented 2 years ago

First of all, sorry if this question has already been asked or answered, but I have searched as much as I could and found nothing.

I've been testing automerge for a couple of weeks, to create a cooperative text editing tool (google docs style). For this I have created a server with flask and websocket and for the frontend I have created a small application with Angular.

The execution flow I am testing is the following:

  1. A client connects to the web and its state is initialized, its sync message is generated and sent to the websocket so that everyone else knows that it has logged in.
    this.localdoc = Automerge.from({content: 'example'});
    this.localSyncState = Automerge.initSyncState();
    let [s1, m1] = Automerge.generateSyncMessage(this.localdoc, this.localSyncState);
    this.sendToWebSocket(m1);
  2. Any user that connects, apart from initializing its state as indicated in point 1, also connects to the websocket so that it listens to all the notifications that arrive to it. And they are synchronized until we receive the null of the generateSyncMessage.

    // m1 is the message sent from another client and received by us.
    // We receive this message and update the one from our current client.
    let [doc2, s1, patch] = Automerge.receiveSyncMessage(this.localdoc, this.localSyncState, m1);
    let [s2, m1] = Automerge.generateSyncMessage(this.localdoc, this.localSyncState);
    if(m1!=null)
    {
      this.sendToWebSocket(m1);
    }
    
  3. When a user types something in the textArea, a .change is made to the automerge doc. And the change is sent to the websocket to be received by the other users, who are connected.

Suppose I open 2 browsers with empty textareas. In this case, when I write in one of them, the message is sent to the other one without any problem. With text in these 2 textAreas already synchronized, if I open another new browser, it synchronizes and receives all the changes without any problem and everything continues to work perfectly.

My problem comes when one of the browsers is closed. Let's suppose browser 2 closes. And I continue writing in browser 1, it continues sending the changes without problem to browser 3. But the moment I open another browser it does not receive the changes and the server is saturated receiving messages non-stop in a loop.

As I understand it, the problem comes because one of the nodes has been disconnected and the Sync messages continue to receive the header of this one and the new node tries to receive its changes and not finding the status, it remains in a loop. Can it be?

I have been reading and I do not understand what is the correct procedure to indicate that a node is disconnected and not to take it into account.

thank you for reading this tome. And thank you for this wonderful library!

ept commented 2 years ago

Hi @joseluratm, sorry for the delayed response. Happy new year!

The sync protocol (generateSyncMessage and receiveSyncMessage) is designed to sync up the local user with one specific remote user. In your case, there are two remote users: browser 1 first talks to browser 2, and then later talks to browser 3. This will not work, because each pair of communicating browsers will need its own sync session.

There are a couple of ways you could handle this:

  1. Run Automerge on the server as well as the browsers, and run the sync protocol only between a browser and the server. This means the server does not forward sync messages from one browser to another browser, but instead the server calls receiveSyncMessage and generateSyncMessage itself. The server will need a separate syncState for every browser that is connected.
  2. Have the server forward sync messages from one browser to another without processing them on the server, but include a unique ID for each browser. Each browser then needs a separate syncState for each other browser it communicates with.
  3. Don't use the sync protocol at all, but have the server distribute encoded documents (as produced by Automerge.save()) or individual changes (as produced by Automerge.getChanges() or Automerge.getLastLocalChange()) instead.
joseluratm commented 2 years ago

don't worry about the delay! Thank you very much for your reply. I have to see how to rethink the problem but with your advice everything has become much clearer.

Thank you very much, and happy new year to you too!