Collaborative editing

[davolokh]

@yatskevich and me did a research and tried to design an appropriate architecture for supporting collaborative editing. There are some key points with explanation below.

The main idea is to implement a kind of Redux-like behavior within Slate. <Editor store={Store} … /> The goal of having Store is ability to have clear separation and understandable flow.

UseCase:

• User press key (types symbol)
• Action is generated
• Action is dispatched to Store
• Store updates State
• setState called
• Editor re-rendered

This way allows us to have ability to manage operations in any suitable way (CRDT, OT, etc.) within Store.

Let’s see how to handle text paste operation in more details:

1. Action pasteText dispatched to Store (data: {position, text}) No need to get current state, call transform() and return new state. Just dispatch an Action.
2. Store has a Reducer for handling such an Action If we’re talking about ColloborativeStore - one type, if about SimpleStore - another one. We can even have few implementation of CollaborativeStores (CRDT and OT).

CollaborativeStore:

• Splits pasteText operation into a set of atomic insertCharacter operations
• Applies operations to State
• Translates this operations set to server
• Listens for updates from server (i.e. smb else is editing this doc and generates the same type of operations set)

SimpleStore:

• Applies operations to State (no need to split to atomic operations)

In our mind such architecture allows Slate to be more flexible and extensible. But this is not a completely analyzed proposal - just an idea.

And a few pictures for clarification.

Workflow Chart:

Implementation example

pastText in depth:

P.S. Possibly, Store should send Transfroms (not Actions) to Server and get Transforms back.

@ianstormtaylor: Could you please take a look at this and share your opinion?

[ianstormtaylor]

I've just added a #collaboration channel to Slack for anyone who wants to discuss this there.

[jeremysf]

Looks like automerge now has explicit support for text strings:

https://github.com/automerge/automerge#text-editing-support

[ghost]

Any update on the status of a collaboration plugin?

[linonetwo]

I think we can try automerge now, some example here:

const Automerge = require('automerge');

// init
const doc1a = Automerge.init();
// another peer
const doc2a = Automerge.init();

// add something
const doc1b = Automerge.change(doc1a, 'Initialize with empty slate state', doc => {
  // from https://docs.slatejs.org/walkthroughs/saving-to-a-database
  doc.note = {
    document: {
      nodes: [],
    },
  };
});
// changes that should sent to peer
let changes = Automerge.getChanges(doc1a, doc1b);
const doc2b = Automerge.applyChanges(doc2a, changes);

// add some content to slate and serialize to JSON and add to automerge
const doc2c = Automerge.change(doc2b, 'Adding some text', doc => {
  // from https://docs.slatejs.org/walkthroughs/saving-to-a-database
  doc.note = {
    document: {
      nodes: [
        {
          object: 'block',
          type: 'paragraph',
          nodes: [
            {
              object: 'text',
              leaves: [
                {
                  text: 'A line of text in a paragraph.',
                },
              ],
            },
          ],
        },
      ],
    },
  };
});
// sent change to doc1's client
changes = Automerge.getChanges(doc2b, doc2c);
const doc1c = Automerge.applyChanges(doc1b, changes);

// change some text deep inside document
const doc1d = Automerge.change(doc1c, 'Rewrite line', doc => {
  doc.note = {
    document: {
      nodes: [
        {
          object: 'block',
          type: 'paragraph',
          nodes: [
            {
              object: 'text',
              leaves: [
                {
                  text: 'Rewrite this line.',
                },
              ],
            },
          ],
        },
      ],
    },
  };
});
changes = Automerge.getChanges(doc1c, doc1d);
const doc2d = Automerge.applyChanges(doc2c, changes);

// we can see the change is done correctly
JSON.stringify(doc2d);
JSON.stringify(doc1d);
console.log('`````');

// see the history
const historyPrinter = state =>
  `[${state.change.message}]: ${JSON.stringify(state.snapshot.note.document, null, '  ')}`;
const state1 = Automerge.getHistory(doc1d).map(historyPrinter);
const state2 = Automerge.getHistory(doc2d).map(historyPrinter);
console.log(
  `finalState1: ${state1}\n\nfinalState2: ${state2}\n\n finalState1 === nfinalState2: ${JSON.stringify(state1) ===
    JSON.stringify(state2)}`
);
console.log('`````');
// this can be saved to database at any time
const serializedContent = Automerge.save(doc1d);
console.log(serializedContent);

Parts of output:

changes = Automerge.getChanges(doc1c, doc1d);
[ { actor: 'fca6a267-8317-4471-a8ed-71267e7d5779',
    seq: 2,
    deps: { '46ede060-927c-4ddb-9224-a9c654c3d42c': 1 },
    message: 'Rewrite line',
    ops:
     [ [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object] ] } ]

// JSON.stringify(doc1d) is the same
JSON.stringify(doc2d);
'{"_objectId":"00000000-0000-0000-0000-000000000000","note":{"_objectId":"bc317181-4521-416b-b8bd-e4c5dfbd1b1f","document":{"_objectId":"6ba1520d-d1e9-4a89-9c09-6ca7be74792b","nodes":[{"_objectId":"2871dbd4-1a40-4d1d-9235-5505a9e76b0c","object":"block","type":"paragraph","nodes":[{"_objectId":"9ac26407-f32c-45cb-ad31-19726574a370","object":"text","leaves":[{"_objectId":"3e3d385c-2cb4-4e3d-b0c9-c0831d954656","text":"Rewrite this line."}]}]}]}}}'

finalState2: [Initialize with empty slate state]: {
  "nodes": [],
  "_objectId": "58274c9f-67af-4800-80c1-ce10f337a7fb"
},[Adding some text]: {
  "nodes": [
    {
      "object": "block",
      "type": "paragraph",
      "nodes": [
        {
          "object": "text",
          "leaves": [
            {
              "text": "A line of text in a paragraph.",
              "_objectId": "2c65869d-6d31-4d04-a6ed-c470a0f00775"
            }
          ],
          "_objectId": "1f0a10a3-287b-47ab-8041-72caa005f1d2"
        }
      ],
      "_objectId": "ce238a19-a36a-4e3e-adc6-63f81d6d1fcd"
    }
  ],
  "_objectId": "420b8b81-c506-4d61-959b-9ef53aab80b9"
},[Rewrite line]: {
  "nodes": [
    {
      "object": "block",
      "type": "paragraph",
      "nodes": [
        {
          "object": "text",
          "leaves": [
            {
              "text": "Rewrite this line.",
              "_objectId": "3e3d385c-2cb4-4e3d-b0c9-c0831d954656"
            }
          ],
          "_objectId": "9ac26407-f32c-45cb-ad31-19726574a370"
        }
      ],
      "_objectId": "2871dbd4-1a40-4d1d-9235-5505a9e76b0c"
    }
  ],
  "_objectId": "6ba1520d-d1e9-4a89-9c09-6ca7be74792b"
}

 finalState1 === nfinalState2: true

[jasonphillips]

I'm just going to drop into this discussion the recent work by Atom / Xray (the latter being a future replacement for Atom by its team, with the backend in Rust, and capable of running entirely in a browser via WASM). They have implemented a robust CRDT approach that is nicely optimized and thankfully allows strings at the lowest level, so it doesn't require each character to have its own key. Earlier in this conversation, it was said that CRDT doesn't have many complete examples in the wild, but it looks as if Xray is poised to change that in a very high-profile way.

An earlier overview is in this presentation: https://www.infoq.com/presentations/crdt-tachyon-collaborative-editing

The newer XRay repo is here, but you'll have to read Rust to get into the string buffer logic: https://github.com/atom/xray

(That project also raises the option of some future slate-like library deciding to move all its data representations and operations logic into Rust as a wasm bundle running concurrently to the javascript code that handles DOM and events... the performance gains might prove substantial.)

[aslakhellesoy]

@jasonphillips is the CRDT you're referring to Teletype-CRDT or something else?

Teletype-CRDT works on a sequence of characters (strings), and cannot be used for rich text, which is represented as a tree.

[jasonphillips]

It's a variation / continuation of their ideas from Teletype, but significantly refactored now (and now written in Rust) as the core logic of their future editor. As-is, it doesn't fit a rich text scenario -- but their work on CRDT is still pretty useful it seems to me, particularly in how they reason about & optimize problems of finding correct fragments without costly offsets etc.

(the presentation linked is from many months back, but their recent work is in the xray repo)

[vshia]

This is still (very much) in the prototype phase. Nathan and I have been working on a bridge between Slate and Automerge. It converts Slate Operations to Automerge operations (and vice versa) to use Automerge's CRDT algorithm/library to handle merge conflicts and synchronize changes between clients.

Currently, the work only supports plain text and lists (using slate-edit-list). It doesn't support Marks yet (though attempting to begin something in this branch).

Let us know what you think. There is still much to do, some of which is listed in the README, but wanted to get this out there for the community to look at. As a side note, there may be a few unforeseen bugs from the recent Slate update to 0.34... if you come across any, let me know.

[Coder1400]

Hey all, and awesome work on Slate!

Now that slate has been made OT ready, I began to look at integrating the Slate editor with Firepad (Serverless firebase driven collaborative editing). Some of the contributors over at Firepad mentioned that it's simply a matter of creating an adapter for Firepad - so it shouldn't be too difficult. It would be nice to allow people to embed a collaborative version of Slate through Firepad with just a couple lines of JS. The monaco editor was integrated into Firepad pretty recently - seemed like a fluid process.

I wanted to begin development pretty soon and was wondering if anyone is interested? Don't know much about OT and some extra hands would be awesome.

Thanks :)

[philcockfield]

Is the "syncing operations" sample the reference example for the work that came out of this issue? Trying to figure out what best to study to get to grips with this area.

https://www.slatejs.org/#/syncing-operations

Thanks

[majelbstoat]

@philcockfield: not exactly. it demonstrates synchronisation of low-level slate operations, but only in the same client. none of the more complex part of merging operations is handled in that example. in practice there would be a bunch of sophisticated merging code on the server to handle distributed collaborative editing.

[bryanph]

Instead of implementing operational transform, perhaps we can start with something simpler, like block-level locking, which locks the block for other users when the user requests a lock on the block.

[alexluong]

Is there any new updates on this issue? Have you made any attempt with FirePad, @Arken94? My company is looking at implementing real-time editor in the upcoming months, so I'd be happy to help with the research and experiment.

[meetbryce]

@alexluong I personally ended up opting for https://github.com/ProseMirror/prosemirror

[rahul1995]

@alexluong I personally ended up opting for https://github.com/ProseMirror/prosemirror

Isn't developing with Prosemirror a little tedious as compared to Slate?

[mitar]

Learning curve is steeper, but results are amazing.

[philcockfield]

@mitar - your comment "Learning curve is steeper, but results are amazing." refers to Prosemirror?

[mitar]

Yes.

[mitar]

I do not think this is a suitable place for such questions. Please turn to ProseMirror discussion forum and/or read through its documentation and examples.

[LionsAd]

I looked into SlateJS - Great work on the API. I really like it.

Here is some information that should be helpful to implement collaborative editing:

• Transactions / Transformations / Operations are only one part of the problem.
• You also need unique-IDs in the schema

While all the world is still implementing OT I would not suggest that approach.

OT's strength is text editing, but not rich text editing. There are numerous head-aches to make OT work with rich text editing and while ot-types/rich-text has come a long way, it still feels wrong (from an Architectural perspective).

CRDT on the other hand just makes sense [as implemented by yJS]:

You give every character within your distributed system a unique ID and you convert all transformations to operate not on ranges, but instead on those IDs.

Here is a blog post that can explain that better than I ever could:

https://conclave-team.github.io/conclave-site/

The summary is:

CRDT is modifying distributed state based on unique identifiers, while OT is transforming transactions.

Of course it is not ideal or performant to give every character it's own ID, but the way to solve that is to merge things together like yJS already does.

https://github.com/y-js/yjs/blob/master/README.v13.md#yjs-crdt-algorithm

So if one wanted to create a CRDT-ready data structure instead of using a pre-build CRDT type and syncing back and forth to it, then adding those IDs [they need to use a generator function provided e.g. by yJS] and a way to transform operations to work on those IDs is the way to go.

[steida]

Agree OT is no go. https://martin.kleppmann.com/2018/05/24/j-on-the-beach.html explains it nicely.

I checked yjs several times and I prefer https://github.com/automerge/automerge

This is a little bit obsolete, but the idea is awesome https://github.com/ept/slate-automerge/tree/master/src

[LionsAd]

Thanks - for our cases yJS worked performant and well.

What problems did you have with it?

BTW. Did you mean?

https://github.com/humandx/slate-automerge

The other link was not much to see.

[justinweiss]

Either OT or CRDT come with significant tradeoffs vs. the other, and whichever will work will depend a lot on your architecture -- I think Slate being agnostic about which to use is the right choice.

If you need peer-to-peer collaborative editing, you don't have much of a choice -- CRDT is probably the only way. On the other hand, if you have a central server, OT-based collaboration for Slate is straightforward (if time-consuming) to build. I haven't found a way to represent the intent of things like splitting nodes, merging nodes, and moving nodes using CRDTs. It might be possible to encode that behavior into the CRDT, but then they start to look a lot like OT :-). With OT, preserving that intent is easy, and easy to change. As interesting as CRDTs are, in my experience, rich text editing doesn't seem like a great fit -- though I'd love to be proven wrong. Plaintext with CRDT, on the other hand... ❤️

If you were going to build CRDT-based collaborative editing in Slate, this paper is the most interesting approach I've seen.

[majelbstoat]

Just to say, OT is clearly not a no go, and is also clearly not unsuitable for rich text because OT is used for Google Docs collaboration. @justinweiss's point about nodes is also well-taken.

[LionsAd]

Longer response to the paper OT vs. CRDT in real applications here:

https://gist.github.com/LionsAd/19673619c2fa438e475ddca9aa2841b3

@majelbstoat Yes, but the question is: How complex does it need to be to achieve that? I got really scared by the tales of the CKEditor 5 people.

@justinweiss The only tradeoff that CRDT has is the tombstones and the lookup tables needed to find the position of the IDs, but CKEditor 5 also needed a graveyard for their OT implementation. I also outlined some thoughts in my linked post that epoch based GC with a central server will work well for removing the tombstones and hence is not a problem in practice.

But now as a conclusion I think that CRDT vs. OT is a pure theoretical question and does not even matter in practice for our modern JSON-model based editors.

The thing is always the same (OT vs. yJS) as example:

• Convert JSON model to plain-text / YXmlFragment
• Apply OT / CRDT operations to converge the state
• Convert plain-text / YXmlFragment back to editor state using either diffs, complete loading, or object lookup + change transformation

I am not seeing the overhead of either OT nor Y here being any different.

And in fact even implementation wise - let's assume we have rich attribute changing operations it is the same conceptually:

• $table->setRows(3)

is converted to:

• Within the OT-plaintext / Y-Model:

• Find the (serialized or deep nested) table object

• Set the attribute 'rows' to '3'

• Sync that change

• Convert model back

But we can do better!

But it can be kept with both CRDT / OT also as a rich operation:

• setAttribute('table', $position_or_id_of_table, 'rows', 3);

and again within OT the start_position of the table would be changed in the operation, while in CRDT it would just reference the ID of the table.

If the table was deleted then the OT operation would be removed as invalid, while the CRDT operation would be applied to the tombstone (or rejected as well as the object is invalid).

Here is a little bit of a difference of OT and CRDT in case of undo, which should preserve the state done while the table was not there anymore:

With CRDT if the table is restored by an undo operation it would have the change made on the tombstone [if that was chosen], but with OT the server would need to keep track of invalid operations and apply them when the undo operation makes that operation valid again (feels harder, but doable).

So the simplest to optimize editors is to allow frameworks like either Y or OT to be able to directly find the objects on which to execute the state change (and it does not matter if that is position or ID based).

One possibility could be a hash table of unique_ids to the immutable objects. Another could be to keep track of all generated objects within an array and having each object know it's ID.

The lookup via plaintext-OT is then as simple as to find the ID within the object and if that is always the first property the serialised structure will always start with e.g.:

{'id': 1234...

and then the object can be loaded and the state change applied to it.

Note: These are just optimizations to avoid the whole diff/reload approach and keep rich object operations where ever possible. And that overall matches more how react works and the original intent of slate.

---

I am not understanding why node splitting is a problem if you could convert the editor JSON to a plaintext model as well and apply plaintext-CRDT then, where it seems to be simple.

Because as we have a 1-1 isomorphism between both models if it is simple in one model, there should be a function that makes it possible in the other model as well.

I still think rich operations should be what is send and applied and if we mix OT and CRDT - who cares as long as its simple ;).

[LionsAd]

Okay, node splitting again:

So the famous example:

• User A makes a paragraph node into a list
• User B splits the paragraph in two paragraph nodes

I am not seeing why this is a problem with CRDT:

The paragraph consists of a sentence of e.g. 42 characters:

Lets assume the client ID is 4 and the counter is 100 and all was inserted by this client:

The range then to apply the list operation is then: 4-100 to 4-142

makeList('4-100', '4-142')

If the paragraph is split by client 5 at e.g. 20 the IDs of the characters (including the new line / paragraph break) are:

4-100...4-120 5-0 4-121...4-142

The makeList operation will still be applied to all characters and hence to both paragraphs.

If it comes first, the whole 4-100...4-142 is made into a list and if it is then split at 4-120 the 5-0 is still inserted correctly.

What am I missing? As long as every character + rich object has it's own ID ranges can always be found.

Moving is a problem yes as it's delete + insert in CRDT and leaves a lot of tombstone records, but it's not trivial in OT either as far as I've understood.

[LionsAd]

I finally understood what the real problem of OT / CRDT is [what the CKEditor guys have talked about] and why those simple cases pose such a problem in Editors:

• Editors use really low level operations, so instead of makeList() / removeList() everything is text based.

e.g. in slate making a list is the following operations:

0: {object: "operation", type: "set_node", path: Array(1), properties: {…}, newProperties: {…}, …}
1: {object: "operation", type: "insert_node", path: Array(1), node: {…}, data: {…}}
2: {object: "operation", type: "move_node", path: Array(1), newPath: Array(2), data: {…}}
3: {object: "operation", type: "move_node", path: Array(1), newPath: Array(2), data: {…}}

This then leads naturally with automerge / CRDT, but probably also naive OT implementations to conflicts, because they only operate on text or JSON, but not on semantic structures.

The case for high level state changing operations

This explains a lot of confusion of me, because I see the text editor universe as a set of high level commutative, inversable and potentially idempotent operations that operate on objects to change state (which is similar to what the OP proposed using redux operations).

In addition they need to change text, but that is a solved problem.

Hence I have two things:

• Internal state (not synchronized, but used for the editor model to display)
• External state which is a flat projection of the DOM model to apply operations to (synchronized via high level operations)
• High level (CRDT) Operations that transform this internal state (synchronized across the wire)

The advantage of a high level operation (as long as we can prove that it is commutative and inversable) is that it should be more user intent preserving:

• If I make a list out of a selection (implicit or explicit) I have two "identifiers" that are showing the range of the list operation

• If someone changes something within this range, I still apply the list operation to this range

• If I undo it and someone has continued to write more list items, this will undo the whole list operation, not just the case where I did that

• If I redo it, then again it would continue to be applied on my original range (it's the best we can do)

In many cases even the more naive prosemirror reconciliation collab approach would be able to transform a selection to be user intent preserving on this high level operation.

Just when receiving such a high level operation would the editor change the internal state in a way that it makes the list, similar to how the DOM is rendered independent of the internal editor model.

Open questions:

• What happens when two users make a list, one undo-es it? [as the operation was on the non-list state originally, it's likely it would just un-list it as both made the same change (idempotent)]

• What happens if there is an object in between, which splits the list as it cannot be in a list item per schema? (The auto-schema would solve that by splitting the list automatically again once undo / redo of makeList)

• When happens if a user intentionally split the list, now someone undo's and redo's it? [even if the list would no longer be split but be just a list-item again, that would be 'acceptable' in most circumstances -- Google Docs handles it worse as undo does nothing in the conflict case.]

Thinking of that, maybe it would be possible to model this as:

{list}Item 1 Text...{no-list}...some more...{no-list-end}Item 3{list-end}

which would be intent preserving, but then the problem is that just {list}{list} would be overriding {no-list}, etc. which makes things really complicated.

Splitting the undo-make-list operation obviously could work on the new identifiers (which is more OT obviously), but let's assume a client that does never read new state, but still pushes out his state changes (possible in CRDT p2p).

And all it does is: undo / redo / undo / redo the makeList operation [or just send makeList / removeList / makeList / removeList / ... alternating on the same range of identifiers].

A possibility to design a split of a range would be to have a splitList operation.

In other words:

A given range in the context of 'list' is mapped to two ranges instead.

Maybe ...

I can now see the merits of quill's delta format.

Okay, let's assume we model the splitting of the list as just removeList(), then it's not longer commutative, because:

• makeList(0,10)
• removeList(4,5)

and

• removeList(4,5)
• makeList(0,10)

do not have the same result, which means currently the makeList / removeList are not commutative on ranges, the same is true for 'marks'.

The question then is: Is it possible to derive a set of high level operations that are user intent preserving AND have the CRDT properties?

TBC ...

---

Okay, I understood an assumption I made:

• The removeList is only a meaningful operation if there is a list to remove (e.g. as a user I must observe a list to be able to make a removeList operation)

• Lets assume there is a list from:

3 - 6

One user removes the list from 4-5 and leaves just positions as lists (3,6), another user increases the list from 0 - 10 (he has observed the list from 3-6).

User A does: removeList(4,5) User B does: makeList(0,10), which is transformed to: makeList(0,2) + makeList(7,10)

Regardless in which order the operations come in, they are non-conflicting.

However when User B had not yet seen the state, then this does not work -- so the OT approach of having the server dictate to first add all remote operations is needed here as well, which I think is okay as it has several other useful properties.

Still the intent of the user might have been to just do a list from 0-10 - regardless if someone else had removed the list or not.

So while the result is nicer it's still in conflict.

And the same problems are true for marks as well: Adding is easy, removing is hard.

However the same is true for CRDT in general: Adding is easy, removing is hard and a tombstone approach is used for removing, so maybe here a tombstone approach needs to be used as well?

TBC ...

---

So how does Google do it?

In Google marks are at least in my testing independent structures that operate on ranges in the document [maybe on a per paragraph basis?] and that are transformed to extend or shrink when the range changes. [at least in the conflict case of mark-bold + split node Google Docs correctly made the whole thing bold -- no idea what happens on removal of mark-bold within the middle]

However lists / list items are different from that and seem in Google Docs to be separate nodes:

Google applies the 'makeList' operation to one node, but not the other.

This implies to me that when Google is splitting a node, that one is kept and the text moved to another node.

---

Researched this some more and internally it seems the operations are converted that lists have both a toggle list thing, but also a depth.

Undo-ing the operation to make the list from 0-10 then re-doing it leads to the funny state of a list that is present for all 4 items, but has a depth of 0 (no-list).

[LionsAd]

I thought some more about this and also researched Google Docs some more:

• A very simple (inefficient, but should be working) model would be to have each character be the main object of the document (therefore it is way more like text-CRDT / text-OT)

Instead of having:

{'some text', marks: ['bold']}, {' that continues here'}

You just have:

'some text that continues here'

And then you have an object / position map, that maps 's' => {'bold': true}, 'o' => {'bold': true}, ...

A paragraph also is just an 'enter' like in normal text with contextual information.

There are also 2 different styles in Google Docs:

• Text styles [applies to characters]
• Paragraph styles [apple to paragraph objects]

And then there is a table, which can contain other text objects, which again have text styles and paragraph styles.

By removing all the nice things that make the editor model so nice to work with (rich-deep, dom-like things) in the model used for CRDT / OT synchronization, the implementation gets simpler.

Every character has it's text styles.

Every paragraph [ends with enter] has it's paragraph styles and indent is just a counting property, list_style is just a enum property, left / right / blockquote is just a style property on the paragraph.

This brings you 90% to Google Docs.

The advantage of character based styles and whole text operations is that the whole conflicts that lead to incomplete node_split or conflicts with duplicated text are avoided.

Last change wins - for properties - also is closer to user intention usually.

It is also much closer to normal OT / CRDT.

For rich text things like tables there would still be conflict, but that is okay -- because the HUGE problem with lists + paragraphs + markers is that they can be converted into each other and lead to node splits, etc.

By not having that problem in the first place, it is mostly avoided.

[sepehr500]

So just to clear the air, is Slate js ready for collaborative editing right now, and if yes, has anybody actually done it in production?

[tommoor]

Yes. At least two folks have separately achieved it in a production environment according to conversations in the Slack group, unfortunately neither has open sourced their work.

[justinweiss]

Yes, at Aha!, we have built a collaborative editor based on Slate. We use Operational Transformation on the Slate operations, which meant writing around 100 transformation functions -- luckily, most were pretty straightforward and could share a lot of the same code.

Once you have that layer, you can also use OT for collaborative cursors and collaborative undo / redo, which is nice. As usual, the details are hard to get right, but it's definitely possible in Slate. Permissions, snapshots, integration with other systems, etc. make things much more complicated :-)

I wrote an article about creating a collaborative editor in general, which goes into most of the basics you'd need to know. It's based on a talk I gave at RailsConf, so the examples are in Ruby, but the ideas translate to whatever language. The CKEditor collaboration article is also useful for learning about some of the other things you'll need to get right in order to make collaboration work. Their "post-fixers," for example, are probably similar to Slate's normalizers.

[t3db0t]

@justinweiss So....—any chance that y'all are open-sourcing your collaborative editor? Eh? Eh? ;D

[TheSpyder]

There are a couple of partial solutions around for collaboration in Slate, although both seem to have reached a certain point and then stopped. It's quite a lot of work to implement all the necessary transforms.

Mentioned above is CRDT with AutoMerge: https://github.com/humandx/slate-automerge

There's also OT with ShareDB: https://github.com/qqwee/slate-ottype

[t3db0t]

@TheSpyder Awesome, thank you :)

[cudr]

Created plugin for co-editing: https://github.com/cudr/slate-collaborative

Hope, this will help for someone

[gnestor]

I think I speak on behalf of most of us when I say thanks so much for sharing @cudr!! 👏👏👏

It's great to see an open-source plugin implementation. I tried it out and it looks great. Can you share a bit about your take on it: what works, what doesn't work, future work, etc?

[cudr]

@gnestor, thank you!) I test it couple of times, and it works correct together with popular plugins. But i haven't time a lot to check this out at all. It will be great if community helps with this, and give feedback about how it works in real (prod).

Welcome to contribute!

[ianstormtaylor]

This stuff is all now possible with Slate. I'm going to close this issue since there's not much left for core to do, but feel free to keep discussing!

[Tamriel]

@cudr: You combined Slate and Automerge - great! @ianstormtaylor: Can you elaborate a bit? How are conflicts handled?

[TheSpyder]

@Tamriel Slate doesn’t do collab out of the box, it has the api available to implement it.

[amilich]

@cudr any plans to update for 0.5+?

[Immortalin]

https://github.com/Immortalin/slate-operational-transform

[Immortalin]

^for Slate 0.5+

[t3db0t]

Really cool @Immortalin ! Do you expect this repo to receive updates and improvements, or stay as a demo?

[Immortalin]

It will most likely stay updated as long as I continue to use Slate in production. But it is primarily an example/reference. However I would love for Slate to have first party integration with ShareDB like Quill.

[Immortalin]

The operations that are currently missing for the ShareDB JSON1 type is merge node and split node. If Slate can offer first party support, it will make a lot of things easier. My solution of using json0-ot-diff is more of a hack than anything. Running a search algorithm every keystroke is not optimal. Don't get me wrong, it works fine and the lag is not too significant even when pasting large documents. But a lot of cycles are wasted since Slate already has Operation types and we are not reusing them in my example.

[Immortalin]

@ianstormtaylor https://github.com/ottypes/json1/blob/master/README.md

[TheSpyder]

This issue was closed months ago, as far as I know Ian isn't interested in offering first party support for RTC. Just the framework to make it possible.

[cudr]

@cudr any plans to update for 0.5+?

@amilich Already updated

[BitPhinix]

I created a yjs based collaboration plugin with good performance on large documents: https://github.com/BitPhinix/slate-yjs

(Tested on the newest slate version)