Collaborative editing step 2

Goal

Enable collaborative text editing on Text and Area items like one does on Google Docs. Additionally, by applying the Quill editor, we get nicer text-editing features, such as automatic bullets.

Status

As implemented in #240 , collaborative editing can now be enabled by signed in users when creating a new board. After some experience in production, the opt-in flag should be removed and collborative editing be the default. However, there's stuff to do before that. This Issue represents the work to be done.

Must-have TODO items

[x] Text align (vertical/horizontal)
[x] Add environment variables for controlling collaborative editing (false, true, opt-in)
[x] Copy-paste fix: when duplicating container, contained item text get duplicated
[x] Playwright tests (create text, reload, change, reload, use two clients, clear indexeddb and reload...)
[x] Hiding does not seem to work properly (not probably related to CRDT)
[x] Cloning a board with CRDT does not seem to copy CRDT content
[x] Playwright tests for offline mode (two users go offline, make edits, go online again)
[x] Performance testing using perf-tester.At least hundreds of CRDT text areas can be created effortlessly it seems.
[x] Collab edit usability: allow placing cursor directly on collab edit text when item is locked by other user
[x] Mobile check: works to some extent.
[x] Fix keyboard handling with Firefox: up/down go to start/end
[x] Include API basic tests in Playwright tests
[x] Use the default CRDT behavior for boards created using the API as well. Include CRDT texts in board, hierarchy, csv requests.
[x] Storage requirement measurements: compare DB storage size between legacy and CRDT boards. In 77cd81b610cccd35b4ab77c79b19ca9470c39493 I took some measurements which indicate that CRDT is slightly more compact. With larger datasets, the difference gets greater in favor of CRDT, as long as in JSON format all events are stored. It's noticeable that JSON events also track authors while this is lost in CRDT
[x] Implement read-only mode for CRDT fields. Now write access is required for establishing the Y.js socket.
[x] Changing boards sends the yjs client in a reconnect loop because mismatching board ids. Seems that an earlier provider keeps trying to connect with the earlier board id. Should deactivate properly!
[x] Move board cloning to server-side
[x] Test collab edit with multiple computers / users in a realistic setting

Nice-to-haves

[x] Link handling is now way inferior to the earlier implementation. I would expect at least: pasting a https link to create a link, and if pasted on top of plain text, make the test a link. Also nice if there was a pop-up link editor for both href and text. Unfortunately there is no readymade solution found for Quill. There's quill-magic-links but it doesn't implement the paste-to-create-link-from-existing-text. Also the quill pop-up editor is tied to toolbar (which we don't want) and doesn't allow text edit (href only). I see that custom quill plugin(s) should be written for this, or maybe even replace Quill with something else. Under progress in #248
[ ] Investigate and fix Mobile Safari issue where cursor goes off after zoom / scroll
[ ] Undo buffer integration. Editor has its own local undo but we should also add the full edit as a global undo item
[ ] Persistence: consider storing CRDT snapshot
[x] Show proper username by the cursor when hovering. Now shows some large number.
[ ] When a user is typing, maybe hide the mouse cursor and show the hover icon beside the caret if possible. Also consider showing hover icons not by the caret, but attached to the text item, like the circles on Google Docs toolbar.
[ ] Fade out idle cursors?
[x] UI: Add a formatting toolbar. Needs some styling - if you now enable toolbar in Quill, it looks broken
[ ] Sharing: split the TypeScript y-websocket server into a separate shared module for others to enjoy

Design

Use 1 Y.js CRDT document per board to represent collaboratively edited text. Each document contains the text property of each collaboratively edited item. The approach can be scaled to include other fields too. Maybe all fields at some point if the approach proves good.
Use Quill editor and y-quill for integrating the editor with the Y.js CRDT
Use y-websocket client for connecting the Y.js CRDT with the server
Port the y-websocket server to TypeScript and into this codebase (see backend/src/y-websocket-server). Customize as necessary
Use Cookies for associating Y.js websockets with Ourboard websockets by sessionId
Persist CRDTs as diffs, that go along the "event bundles" we store as database rows. Consider additionally storing a snapshot of the whole CRDT

Utilizing CRDT more widely

It would be interesting to see if we can switch the whole board state into CRDT. Would likely simplify code. Performance might also improve, but this is something to investigate. Anyway, it makes sense to me to first reap the immediate collaborative editing benefits and then see how far we can go.

raimohanska / ourboard