Support passing clientID to doc transformation utils

[gauravtiwari]

Thanks for this library 🙏

I have been trying to figure out the initial hydration process especially when the editor content is not saved in Y.Doc() format

This PR addresses a missing feature to fully support the template engine approach described here: https://discuss.yjs.dev/t/initial-offline-value-of-a-shared-document/465/13

After this PR

// Server
clientID = userSessionID // (only happens once across a collaboration session)
const contentParser = markdownParser(pmSchema)
const pmDoc = contentParser.parse(markdown)
const ydocTemplate = toBase64(encodeStateAsUpdateV2(prosemirrorToYDoc(pmDoc, clientID)))

// client
const ydoc = new Y.Doc()
// Loaded server side and passed to DOM
Y.applyUpdateV2(ydoc, fromBase64(ydocTemplate))
ydoc.clientID = userSessionID

Before this PR

The prosemirrorToYDoc function uses new Y.Doc(), which assigns a random clientID causing a mismatch:

const ydoc = new Y.Doc() // assigns a random clientID in the browser 
Y.applyUpdateV2(ydoc, fromBase64(ydocTemplate)) // random clientID on the server

// => store.clients does not include ydoc.clientID

[gauravtiwari]

Turns out this isn't an issue anymore and random ids are much better to avoid conflicts

[vojto]

@gauravtiwari Hey, could you share a bit more about your usecase? And how'd you fix your problem?

I'm dealing with a situation where 2 clients can create a document with some initial content, for example "# Fri, May 13th". They could both create the document, and start adding updates locally.

I could do this on both clients: const ydoc = prosemirrorToYDoc(/* doc with '# Fri, May 13th' */). But if I now send my updates to server, each set of updates will be expecting different client IDs on this snippet.

I could create an empty Y.Doc on both clients, and then send my header snippet as update. But when I merge, I'll have the header snippet twice.

@dmonad I know this has been discussed in this thread, but it seems to me my only solution is to initialize document with that snippet, and clientID forced to a constant.

Edit: Replied in the thread - better to continue conversation there.

[gauravtiwari]

Hey @vojto

I have moved away from using yDoc in the end but here is the approach:

on the server side: (using ruby so had to call JS via embedded V8)

const toYDoc = (markdown, username) => {
  const contentParser = markdownParser(pmSchema)
  const pmDoc = contentParser.parse(markdown)
  const ydoc = prosemirrorToYDoc(pmDoc)
  ydoc.gc = false

  const permanentUserData = new PermanentUserData(ydoc)
  permanentUserData.setUserMapping(ydoc, ydoc.clientID, username)
  const versions = ydoc.getArray('versions')

  versions.push([
    {
      date: new Date().getTime(),
      snapshot: encodeSnapshotV2(snapshot(ydoc)),
      clientID: ydoc.clientID
    }
  ])

  return toBase64(encodeStateAsUpdateV2(ydoc))
}

const fromYDoc = (ydoc) => {
  const contentSerializer = markdownSerializer(pmSchema)
  const newYdoc = new Doc()
  applyUpdateV2(newYdoc, fromBase64(ydoc))

  return contentSerializer.serialize(yDocToProsemirror(pmSchema, newYdoc))
}

One important note: please ensure no two clients can create two sets of yDoc so initial yDoc is only created once per collaboration session.

Embed the returned yDoc on initial load using template technique:

<div data-ydoc='loaded from that method above'></div>

Load in the JS editor (for example: Prosemirror)

const ydoc = dataset.ydoc
this.ydoc = new YDoc()
this.ydoc.gc = false
applyUpdateV2(this.ydoc, fromBase64(ydoc))

Hope this helps and Happy to share the complete code if that make things easier for you.