Not synchronized

[DenysPoliarush]

I use a collaboration extension so that many users can work together on the same document, but users connecting to the same document see different content and they are out of sync with the rest.

In this video you can see how one user opened the same document in 4 browsers - chrome, opera, firefox and safari.

At first they all synced fine, but after restarting all pages the safari connected to another document and was not synced with the other browsers.

Then we switched to another document and saw the same picture, safari connected to another document.

Then we switched to another document, and we can see all the browsers were synchronized.

This also happens when different users connect to the same document.

https://user-images.githubusercontent.com/66991536/128873164-564b21a1-eaa2-4c9d-9c70-af41ae0d854c.mov

Please let me know if you need my code.

[tommoor]

There are many different ways this could happen, the bug is just as likely in your application code. You should trace through what safari is doing, which document id it is connecting to, where is the document state coming from ... indexeddb, rocksdb, memory, broadcastchannel?

Yes, code is needed for any meaningful debugging conversation 😄

[hanspagel]

I had this too, see #126. I failed to find the cause and hoped more people would hit this and can contribute debugging information. So yes, your code could help!

[tommoor]

Unfortunately y-websocket has a bunch of still unfixed race conditions, so it's not a surprise that some would have made it across:

https://github.com/yjs/y-websocket/pull/39

Something that immediately looks a little suspect:

https://github.com/ueberdosis/hocuspocus/blob/0a187b35c6b5f1ea0ee327556346b683753e3c10/packages/server/src/Hocuspocus.ts#L177

What happens here when several clients connect in the time it takes for onCreateDocument to resolve a value from persistence? Do they all get their own version of the document?

I believe in createDocument we should check if the document exists in the map before the new Promise and if not, synchronously create the document and add it into the map.

[hanspagel]

Good find!

[DenysPoliarush]

Thanks for the quick response!

I'm using koa and koa-easy-ws. Here is my code

const { debounce } = require('debounce');

const { Server } = require('@hocuspocus/server');
const { Logger } = require('@hocuspocus/extension-logger');
const { RocksDB } = require('@hocuspocus/extension-rocksdb');
const { Throttle } = require('@hocuspocus/extension-throttle');
const { TiptapTransformer } = require('@hocuspocus/transformer');

const extensions = require('../libs/tiptap/extensions');

const { getSchema } = require('@tiptap/core');
const { DOMParser } = require('prosemirror-model');
const { JSDOM } = require('jsdom');

let debounced;

const server = Server.configure({
  extensions: [
    new Throttle({
      throttle: 20,
      banTime: 1
    }),
    new Logger(),
    new RocksDB({
      path: './tiptap-store'
    })
  ],

  async onCreateDocument (data) {
    const fieldName = 'default';

    if (!data.document.isEmpty(fieldName)) {
      return;
    }

    const project = await ProjectService.getById({
      id: data.context.projectId
    });

    let prosemirrorJSON = project.document;

    if (!prosemirrorJSON) {
      const { document } = (new JSDOM(project.html.trim())).window;

      const schema = getSchema(extensions);

      prosemirrorJSON = DOMParser
        .fromSchema(schema)
        .parse(document, { preserveWhitespace: true })
        .toJSON();
    }

    return TiptapTransformer.toYdoc(prosemirrorJSON, fieldName, extensions);
  },

  async onConnect (data) {
    const { requestParameters } = data;

    const token = requestParameters.get('token');

    const user = await UserService.getUserByToken(token);

    return {
      userId: user.id,
      token: token
    };
  },

  async onChange (data) {
    const save = async () => {
      try {
        await ProjectService.update(someData);
      } catch (error) {
        console.log(error);
      }
    };

    debounced?.clear();
    debounced = debounce(() => save(), 1000);
    debounced();
  },

  timeout: 30000
});

router.get('/:projectId', async ctx => {
  const projectId = ctx.params.projectId;

  try {
    if (ctx.collaborationWS) {
      const documentName = projectId;
      const ws = await ctx.collaborationWS();

      server.handleConnection(
        ws,
        ctx.request,
        documentName,
        {
          projectId: projectId
        }
      );
    }
  } catch (err) {
    ctx.bad(400, err);
  }
});

[tommoor]

What happens here when several clients connect in the time it takes for onCreateDocument to resolve a value from persistence? Do they all get their own version of the document?

I created a test for this and I can't reproduce in that environment, I do believe it's possible probably on a server under heavy load though – so I will put forth an improvement PR anyway. Based on that it's unlikely to be the bug at cause here.

[hanspagel]

@DenysPoliarush Can you update to the latest version and test again? 🙌

[DenysPoliarush]

Hi @hanspagel.

I have updated to the latest version and it works much better than before. Now it connects to the same document.

But sometimes it still doesn't sync. This happens quite often when I switch between my documents or just reload the page.

https://user-images.githubusercontent.com/66991536/133602347-ebe98791-3e8d-42c8-885a-65a73a599515.mov

And after update the @hocuspocus/provider to version 1.0.0-alpha.15, I started getting an error in the console Uncaught (in promise) undefined

[DenysPoliarush]

Hi @hanspagel, is there any news?

[DenysPoliarush]

Hi @hanspagel , do you have any updates here? Perhaps this has already been fixed? Let me know and I will update to the latest version and check it out. Thanks!

[hanspagel]

Hi @DenysPoliarush! Sorry for the long wait. I still can’t reproduce the issue. But I’ve changed a ton over the time. Would you mind checking the latest version?