ueberdosis / hocuspocus

The CRDT Yjs WebSocket backend for conflict-free real-time collaboration in your app.
https://tiptap.dev/docs/hocuspocus/introduction
MIT License
1.21k stars 117 forks source link

RangeError Invalid typed array length: 110 & TypeError: contentRefs[(info & binary.BITS5)] is not a function #748

Closed danielblignaut closed 9 months ago

danielblignaut commented 10 months ago

Please save me some time and use the following template. In 90% of all issues I can't reproduce the problem because I don't know what exactly you are doing, in which environment, or which y-* version is responsible. Just use the following template even if you think the problem is obvious.

Checklist

Describe the bug

Our Sentry instance is intermittently receiving this RangeError ->

<anonymous> in new Uint8Array
/node_modules/yjs/node_modules/lib0/buffer.js in Module.createUint8ArrayViewFromArrayBuffer at line 24:84
/node_modules/yjs/node_modules/lib0/decoding.js in readUint8Array at line 104:23
/node_modules/yjs/node_modules/lib0/decoding.js in Module.readVarUint8Array at line 119:45
/node_modules/yjs/dist/yjs.mjs in UpdateDecoderV1.readBuf at line 794:43
/node_modules/yjs/dist/yjs.mjs in Array.readContentBinary at line 8432:64
/node_modules/yjs/dist/yjs.mjs in readItemContent at line 10014:76
/node_modules/yjs/dist/yjs.mjs in readClientsStructRefs at line 1400:13
/node_modules/yjs/dist/yjs.mjs in <anonymous> at line 1627:16
/node_modules/yjs/dist/yjs.mjs in transact at line 3354:14
/node_modules/yjs/dist/yjs.mjs in readUpdateV2 at line 1620:3
/node_modules/yjs/dist/yjs.mjs in applyUpdateV2 at line 1715:3
/node_modules/yjs/dist/yjs.mjs in applyUpdate at line 1729:58

For context, in our code, we have code around this exception like so:

const documentRes = await fetch(`https://${env('NEXT_PUBLIC_TIPTAP_APP_ID')}.collab.tiptap.cloud/api/documents/${documentId}?format=yjs`, {
    method: 'get',
    headers: {
      Authorization: env('TIPTAP_API_SECRET'),
    },
  })

  const buffer = await documentRes.arrayBuffer()
  const initialYDocUpdate = new Uint8Array(buffer);

const ydoc = new Doc()
applyUpdate(ydoc, initialYDocument); #this line throws the exception

Following the stack trace, the problematic line seems to come from the underlying lib0 library:


/**
 * Create Uint8Array with initial content from buffer
 *
 * @param {ArrayBuffer} buffer
 * @param {number} byteOffset
 * @param {number} length
 */
//This line is the origin of the error
export const createUint8ArrayViewFromArrayBuffer = (buffer, byteOffset, length) => new Uint8Array(buffer, byteOffset, length)
/**
 * Create Uint8Array with initial content from buffer
 *
 * @param {ArrayBuffer} buffer
 */

I'm unfamiliar with the restrictions around Uint8Arrays but I suspect there's one of 2 things going wrong:

  1. we are receiving a bad response from TipTap for the initial Uin8Array
  2. I am mishandling the response from TipTap when applying the update (there's perhaps an issue in how I'm converting the ArrayBuffer to an Uint8Array
  3. There's a bug in lib0 or Yjs code causing this issue

To Reproduce

Unfortunately I can't reproduce the issue, it seems to happen intermittently and I am not receiving any bad responses from TipTap Although ArrayBuffers / Uint8Arrays are not very human readable in the first place.

Have you seen an issue like this before?

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Environment Information

Additional context

danielblignaut commented 10 months ago

Additionally I've also seen a single request fail with this stack trace and error from the exact same line mentioned above in our source code:

TypeError contentRefs[(info & binary.BITS5)] is not a function
/node_modules/yjs/dist/yjs.mjs in readItemContent at line 10014:76
/node_modules/yjs/dist/yjs.mjs in readClientsStructRefs at line 1400:13
/node_modules/yjs/dist/yjs.mjs in <anonymous> at line 1627:16
/node_modules/yjs/dist/yjs.mjs in transact at line 3354:14
/node_modules/yjs/dist/yjs.mjs in readUpdateV2 at line 1620:3
/node_modules/yjs/dist/yjs.mjs in applyUpdateV2 at line 1715:3
/node_modules/yjs/dist/yjs.mjs in applyUpdate at line 1729:58

which is making me wonder if there's perhaps an issue in some of the Node.JS implementation?

danielblignaut commented 10 months ago

Originally opened the ticket against Y.JS whom referred me here instead - https://github.com/yjs/yjs/issues/599

janthurau commented 10 months ago

weird! Can you send me your app id (either to humans@tiptap.dev, or in Discord), and if possible dates / times at which you get these errors (maybe screenshots vom Sentry)? Will check if I can find anything in our logs then.

jelling commented 8 months ago

Was the cause discovered? We're having the same issue and can reproduce it reliably with our unit test.

janthurau commented 8 months ago

the issue means that the ydoc update is not valid. Some libraries (here fetch) are somehow breaking some parts of it, it works reliably when using axios and specifying arraybuffer as response type (as mentioned in our docs: https://tiptap.dev/docs/editor/cloud#get-document).

jelling commented 8 months ago

For anyone stumbling into this problem - or for search purposes, any other problem with node-fetch and TipTap - support confirmed that this is still an open issue. I ran into other issues that might have been related to node-fetch with TipTap but hard to say for certain.

In the meantime, the best advice seems to be to only use Axios with TipTap. Hopefully, TipTap will re-open this issue and fix it; and update their docs in the meantime to reflect the known issue.