search

ably / ably-js

Javascript, Node, Typescript, React, React Native client library SDK for Ably realtime messaging service
https://ably.com/download
Apache License 2.0
316 stars 55 forks source link

Fix errors when receiving WebSocket data in Vercel Edge runtime and indefinitely creating new connections #1731

Open VeskeR opened 6 months ago

VeskeR commented 6 months ago

When using an Ably.Realtime instance in the Vercel Edge runtime (to avoid https://github.com/ably/ably-js/issues/1732 error replace setImmediate calls in ably-js with setTimeout) - whether within serverless functions or in SSR when rendering the frontend in Edge runtime - the connection to Ably servers sets up correctly. However, we begin to get TypeError: Cannot read properties of null (reading 'length') errors continuously after the connection has been set up. Here's the full error log:

TypeError: Cannot read properties of null (reading 'length')
    at WebSocketTransport.onWsData (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6373:131)
    at wsConnection.onmessage (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6342:31)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:826:20)
    at WebSocket.dispatchEvent (node:internal/event_target:761:26)
    at fireEvent (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12687:14)
    at websocketMessageReceived (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12709:7)
    at _ByteParser.run (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13122:17)
    at _ByteParser._write (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13001:14)
    at writeOrBuffer (node:internal/streams/writable:564:12)
    at _write (node:internal/streams/writable:493:10)
 ⨯ uncaughtException: TypeError: Cannot read properties of null (reading 'length')
    at WebSocketTransport.onWsData (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6373:131)
    at wsConnection.onmessage (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6342:31)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:826:20)
    at WebSocket.dispatchEvent (node:internal/event_target:761:26)
    at fireEvent (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12687:14)
    at websocketMessageReceived (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12709:7)
    at _ByteParser.run (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13122:17)
    at _ByteParser._write (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13001:14)
    at writeOrBuffer (node:internal/streams/writable:564:12)
    at _write (node:internal/streams/writable:493:10)
 ⨯ uncaughtException: TypeError: Cannot read properties of null (reading 'length')
    at WebSocketTransport.onWsData (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6373:131)
    at wsConnection.onmessage (webpack-internal:///(middleware)/./node_modules/ably/build/ably.js:6342:31)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:826:20)
    at WebSocket.dispatchEvent (node:internal/event_target:761:26)
    at fireEvent (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12687:14)
    at websocketMessageReceived (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:12709:7)
    at _ByteParser.run (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13122:17)
    at _ByteParser._write (eval at requireWithFakeGlobalScope (C:\Users\Andrew\dev\ably\ably-js-next-example\node_modules\next\dist\compiled\edge-runtime\index.js:1:657096), <anonymous>:13001:14)
    at writeOrBuffer (node:internal/streams/writable:564:12)
    at _write (node:internal/streams/writable:493:10)

It is coming from the WebSocketTransport.onWsData method at this line. Here, data parameter unexpectedly is null, causing an error. Judging by the frequency of errors above, it is highly likely that this error occurs whenever we receive an {"action":0} heartbeat message from the server.

After some time, we encounter the next error:

00:25:01.012 Ably: Transport.onIdleTimerExpire(): No activity seen from realtime in 25075ms; assuming connection has dropped

Transport.onIdleTimerExpire() is triggered because we haven't observed any activity on the opened connection for the maximum idle interval. This is due to the failure of processing the heartbeat.

The above issues are accompanied by the following strange behavior:

  1. Even though we encounter errors for heartbeats, we do not encounter errors when receiving other published messages. For instance, we can publish a message to a channel and receive it correctly in the Ably.Realtime instance running in the Vercel Edge runtime.
  2. We can successfully publish messages via the open WebSocket connection, despite encountering errors when trying to process a heartbeat.
  3. We continue to open new connections to the Ably server after each Transport.onIdleTimerExpire() process, and previous connections are not immediately closed. Consequently, this results in the Ably.Realtime instance continuously opening new connections to Ably servers and eventually exhausting the maximum connection pool per account. This is the most critical problem in this issue, preventing us from allowing customers to use the Ably.Realtime instance in the Vercel Edge runtime at the moment.

┆Issue is synchronized with this Jira Task by Unito

VeskeR commented 6 months ago

This comment from ably react-hooks repo might be slightly related: https://github.com/ably-labs/react-hooks/issues/8#issuecomment-1701793139, user saw his "max connections" linearly increase (just as we did in this issue) when ably-js was run on the server during SSR

rnbrady commented 4 weeks ago

This may be related to vercel/edge-runtime#983 as that bug is also caused by a heartbeat and also results in a null object being delivered to the message handler.