jamsocket / y-sweet

A standalone yjs server with persistence to S3 or filesystem.
https://docs.y-sweet.dev
Other
414 stars 27 forks source link

Websocket crashing with large updates #217

Open CraigFeldspar opened 5 months ago

CraigFeldspar commented 5 months ago

Whenever using the test y-sweet endpoint, I often get the websocket to crash :

main.mjs:533 WebSocket connection to 'wss://y-sweet.net/p/....(url hidden)' failed: Close received after close

After that, no reconnection happens automatically, the page needs to be reloaded. This happens usually with larger updates rather that short ones. Is there some message size limitation ? If yes, how can we gracefully handle these kind of errors in order to resend the updates more gradually ?

Also, if you need the full URL to analyze logs on your side, please let me know

paulgb commented 5 months ago

Thanks for reporting, Craig.

When you say large updates, do you have a ballpark estimate of how large these updates are? (I can start by generating updates of that size to see if I can replicate it)

CraigFeldspar commented 5 months ago

So I got it to crash with roughly 10k keys updates, each key is a 36-lengthed string, and each value is a record holding 2 keys and 6 numbers as values. I would say this sums up to roughly 150 bytes per key-value pair, times 10k, so 1.5 Mb (before yjs-ysweet internal binary compression)

CraigFeldspar commented 2 months ago

@paulgb I managed to get a simple repro for you :

https://github.com/CraigFeldspar/ysweet-crash

Steps :

I just added to the base color grid example this piece of code in ColorGrid.tsx:

  const sceneMap = useMap<any>('sceneMap')

  useEffect(() => {
    ;(async () => {
      const obj = await (await fetch('./data.json')).json()
      for (const key in obj) {
        sceneMap.set(key, obj[key])
      }
    })()
  }, []) 

data.json is ~7Mb in size

paulgb commented 2 months ago

Thank you! Will take a look.

ilya2204 commented 1 month ago

Any updates? I have this issues too