Real Time Collaboration

[robnagler]

The ngui should allow real time collaboration, like google docs. This is more important than ever due, because we don't want to be send a big blob of data back and forth every time one part of the data changes. With React and the backend, we know when and what parts of the data change, and often, these changes are conflict free. Moreover, we have to know the relationships between the data as defined in the schema (@garsuga was working on adding this to the schema yesterday).

Here are some references I found quickly:

• https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type
• https://fauna.com/blog/distributed-consistency-at-scale-spanner-vs-calvin
• https://www.tiny.cloud/blog/real-time-collaboration-ot-vs-crdt/
• https://github.com/tinymce/tinymce-react

@kevin-bruhwiler we may want to have you work on some of this. It doesn't need to scale so we definitely can have a small "a" in CAP (search for "small a").

We might be able to use a library to support this.

[garsuga]

Doing my own research I found this which seems to use the 'Two-Phase Set' data structure from the wiki you linked. They have some useful thought processes lined out for the kind of text we use. Lots of complicated libraries out there that account for stuff like styling. If we just need character-by-character editing to be consistent then we could probably make our own solution. Do we need this to work with merging copies of the data that were edited 'offline' or are we assuming this will be live editing like Google Docs?

[robnagler]

I was not clear. What we are "editing" is the contents of the sirepo-data.json files. We have at least two replicas: the GUI and the server. If you also count a copy in the job agents (that run the compute jobs), then there are even more. If the data are updated while a compute job is running, the compute job should either be restarted or terminated. The results should not be returned as they do not correspond to the data in the GUI and the server.

Styling is not the issue here. TinyMCE is interesting because it is an open source collaborative editor so we can see how they deal with concurrent updates. And, it seems to be referenced in this space (conflict free replicated data).