request timeout - Githubissues

Madriix commented 2 years ago

I installed meething-mediasoup by recovering the meething-mediasoup that was on my old server, I installed everything on the server and client side with "npm i" and the side seems to crash on average every 6h to 12h with this mistake:

_createConsumer() | failed:Error: TypeError: Cannot read properties of null (reading 'consume')
    at Peer._handleResponse (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:282:18)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:214:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)
    at /home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:323:40
    at processTicksAndRejections (node:internal/process/task_queues:78:11) {
  [stack]: "Error: TypeError: Cannot read properties of null (reading 'consume')\n" +
    '    at Peer._handleResponse (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:282:18)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:214:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)\n' +
    '    at /home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:323:40\n' +
    '    at processTicksAndRejections (node:internal/process/task_queues:78:11)',
  [message]: "TypeError: Cannot read properties of null (reading 'consume')",
  code: 500
}
_createConsumer() | failed:Error: request timeout
    at Timeout._onTimeout (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156:14)
    at listOnTimeout (node:internal/timers:557:17)
    at processTimers (node:internal/timers:500:7) {
  [stack]: 'Error: request timeout\n' +
    '    at Timeout._onTimeout (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156:14)\n' +
    '    at listOnTimeout (node:internal/timers:557:17)\n' +
    '    at processTimers (node:internal/timers:500:7)',
  [message]: 'request timeout'
}
request failed:Error: producer with id "e509edbb-5687-4003-94fe-b71eb325edaf" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "e509edbb-5687-4003-94fe-b71eb325edaf" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "e509edbb-5687-4003-94fe-b71eb325edaf" not found'
}
/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156
                    pReject(new Error('request timeout'));
                            ^

Error: request timeout
    at Timeout._onTimeout (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156:14)
    at listOnTimeout (node:internal/timers:557:17)
    at processTimers (node:internal/timers:500:7)

it could come from where the problem?

OS: Debian 11 (my old OS: Debian 9) my package.json:

{
  "name": "meething-mediasoup",
  "version": "1.0.2",
  "description": "",
  "scripts": {
    "start": "nodemon ./main.js ./lib"
  },
  "dependencies": {
    "mediasoup": "^3.7.9",
    "protoo-server": "^4.0.3",
    "quick-lru": "^5.1.0",
    "stun": "^2.1.0"
  },
  "devDependencies": {
    "nodemon": "^1.19.1"
  }
}

Madriix commented 2 years ago

I just put this in "main.js" to see if it will crash:

process.on('uncaughtException', (err, origin) => {
  console.error(`${parseInt(Number(new Date()) / 1000)} # Serious problem (${origin}) - this should not happen but the meething-mediasoup is still running. ${err.stack}`);
});

lmangani commented 2 years ago

Hey @Madriix could you show the output of npm install on the module directory?

lmangani commented 2 years ago

The issue seems to be with the protoo websockets - are you perhaps running out of ports or is anything happening to the stack previous this snowball?

Madriix commented 2 years ago

Here is the output from npm install (I just deleted the 'node_modules' folder):

debian@ns30700:~$ cd /home/debian/node/mediasoup/meething-mediasoup
debian@ns30700:~/node/mediasoup/meething-mediasoup$ npm i
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

added 417 packages, and audited 418 packages in 2m

14 packages are looking for funding
  run `npm fund` for details

12 vulnerabilities (6 moderate, 6 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
debian@ns30700:~/node/mediasoup/meething-mediasoup$ pm2 stop meething-mediasoup-server
debian@ns30700:~/node/mediasoup/meething-mediasoup$ pm2 start meething-mediasoup-server

--
debian@ns30700:~/node/mediasoup/meething-mediasoup$ node -v
v16.13.1
debian@ns30700:~/node/mediasoup/meething-mediasoup$ npm -v
8.2.0

the current package.json:

{
  "name": "meething-mediasoup",
  "version": "1.0.2",
  "description": "",
  "scripts": {
    "start": "nodemon ./main.js ./lib"
  },
  "dependencies": {
    "mediasoup": "^3.7.9",
    "protoo-server": "^4.0.6",
    "quick-lru": "^5.1.0",
    "stun": "^2.1.0"
  },
  "devDependencies": {
    "nodemon": "^1.19.1"
  }
}

The day before yesterday I tried to put mediasoup last version but it did not start, suddenly I put back "mediasoup@3.7.9" which works.

For ports 2345 and 20000:29999 I would try to investigate. It's an empty Debian 11 with nothing inside except nginx, nodejs, and 6 or 7 nodejs apps with ports, the rest have no ports. The firewall I use is ufw.

And also compared to Debian 9, on this Debian 11 server I made sure to create an AAAA ipv6 dns to the meething-mediasoup server, maybe that's the crash?

lmangani commented 2 years ago

Interesting! We never tested this code with IPv6 so this might well be the cause indeed!

Madriix commented 2 years ago

hum ok I just deleted the AAAA DNS to see and observe for 2 or 3 days to see if it crashes. But I must not count the next 24 hours which would be a risk for the dns spread.

Madriix commented 2 years ago

About 2 days ago I deleted the AAAA DNS that pointed to the meething-mediasoup server and I also deleted this:

process.on('uncaughtException', (err, origin) => {
  console.error(`${parseInt(Number(new Date()) / 1000)} # Serious problem (${origin}) - this should not happen but the meething-mediasoup is still running. ${err.stack}`);
});

The crash still happens

the last time was tonight

here is a preview of meething-mediasoup-server-error__2021-12-18 :

request failed:Error: producer with id "bf5ebee7-15a8-4fed-b87c-6ba91e7c0be6" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "bf5ebee7-15a8-4fed-b87c-6ba91e7c0be6" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "bf5ebee7-15a8-4fed-b87c-6ba91e7c0be6" not found'
}
request failed:Error: producer with id "c8dc2825-df0c-4f8c-955a-91a923de8bc2" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "c8dc2825-df0c-4f8c-955a-91a923de8bc2" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "c8dc2825-df0c-4f8c-955a-91a923de8bc2" not found'
}
request failed:Error: producer with id "2ef4ba2e-9910-4e76-a45b-bed3ccb0e101" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "2ef4ba2e-9910-4e76-a45b-bed3ccb0e101" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "2ef4ba2e-9910-4e76-a45b-bed3ccb0e101" not found'
}
request failed:Error: producer with id "213f87a4-d099-4239-8c9a-1cc685fe3fd8" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "213f87a4-d099-4239-8c9a-1cc685fe3fd8" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "213f87a4-d099-4239-8c9a-1cc685fe3fd8" not found'
}
request failed:Error: producer with id "b8710436-92cc-43bc-906f-9c7cdb7cb23a" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "b8710436-92cc-43bc-906f-9c7cdb7cb23a" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "b8710436-92cc-43bc-906f-9c7cdb7cb23a" not found'
}
request failed:Error: producer with id "2abdf659-66bd-4005-a21f-e45fd90579c0" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "2abdf659-66bd-4005-a21f-e45fd90579c0" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "2abdf659-66bd-4005-a21f-e45fd90579c0" not found'
}
request failed:Error: producer with id "b7332096-7359-4ae9-8677-ef34988a0170" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "b7332096-7359-4ae9-8677-ef34988a0170" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "b7332096-7359-4ae9-8677-ef34988a0170" not found'
}
request failed:Error: producer with id "1a07ddb5-ff8f-4c54-81fa-733106e1b7c3" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "1a07ddb5-ff8f-4c54-81fa-733106e1b7c3" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "1a07ddb5-ff8f-4c54-81fa-733106e1b7c3" not found'
}
request failed:Error: producer with id "8b77576d-2a53-4f39-b462-855e96e658a6" not found
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)
    at Peer.emit (node:events:390:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:390:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:390:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: producer with id "8b77576d-2a53-4f39-b462-855e96e658a6" not found\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:222:17)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:58:12)\n' +
    '    at Peer.emit (node:events:390:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:390:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:390:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'producer with id "8b77576d-2a53-4f39-b462-855e96e658a6" not found'
}
/home/debian/node/mediasoup/meething-mediasoup/node_modules/mediasoup/lib/Channel.js:199
                        sent.reject(new Error(msg.reason));
                                    ^

Error: Consumer not found
    at Channel._processMessage (/home/debian/node/mediasoup/meething-mediasoup/node_modules/mediasoup/lib/Channel.js:199:37)
    at Socket.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/mediasoup/lib/Channel.js:61:34)
    at Socket.emit (node:events:390:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at Pipe.onStreamRead (node:internal/stream_base_commons:199:23)

Maybe the problem is in the following symptoms:

Problem with Debian 11 which is different from Debian 9
On Debian 9, I saw that my network was "eth0" and that on Debian 11 I have an "eno3" - IPv6 is enabled on both dedicated servers with an almost identical secondary dns, the only thing that changes c 'is "eth0" and "eno3"
Maybe it's because of the UFW firewall I use on Debian 11, and it was CSF / webmin on Debian 9
Maybe it's because of the "debian" user

Often the crash is caused about 6 or 8 or 12 hours or 24 hours after it is started.

Madriix commented 2 years ago

Interesting! We never tested this code with IPv6 so this might well be the cause indeed!

I disabled the IPv6 DNS pointing to this nginx proxy, and also I removed the port range in allow/tcp in the firewall (ufw) and I also removed the AAAA DNS on the domain that points to the proxy .

However, I have not disabled the IPv6 of the dedicated server, do you know if it should be disabled for meeting-mediasoup to work correctly?

Where could the problem come from?

cat package.json
{
  "name": "meething-mediasoup",
  "version": "1.0.2",
  "description": "",
  "scripts": {
    "start": "nodemon ./main.js ./lib"
  },
  "dependencies": {
    "mediasoup": "^3.6.25",
    "protoo-server": "^4.0.6",
    "quick-lru": "^5.1.1",
    "stun": "^2.1.0"
  },
  "devDependencies": {
    "nodemon": "^2.0.15"
  }
}

cat client/package.json
{
  "name": "mediasoup-demo-simple-client",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "start": "parcel index.html vidtest.html --no-hmr --port 3010"
  },
  "dependencies": {
    "mediasoup-client": "^3.1.2",
    "protoo-client": "^4.0.6"
  },
  "devDependencies": {
    "babel-core": "^6.26.3",
    "babel-preset-env": "^1.7.0",
    "parcel-bundler": "^1.12.3"
  }
}

Here is the error it returns to me after doing package traffic: and deletions of dns ipv6.. :


    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 1 }, { consumer: 0 },
    ... 8 more items
  ]
}
request:getRouterRtpCapabilities
request:createWebRtcTransport
request:createWebRtcTransport
request:join
protoo Peer "close" event [peerId:Davidbrow]
last Peer in the room left
protoo connection request [peerId:51......, address:/?roomId=g1oeknrxh6l&peerId=Davidbrow&t=1646868383557, room:%s]
existing room stat g1oeknrxh6l Davidbrow {
  peer: 1,
  transport: 108,
  transports: [
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { producer: 0 },
    { producer: 0 }, { producer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    ... 8 more items
  ]
}
request:getRouterRtpCapabilities
request:createWebRtcTransport
request:createWebRtcTransport
request:join
protoo connection request [peerId:51......, address:/?roomId=6vk27vce76c&peerId=Taminya_&t=1646868787987, room:%s]
existing room stat 6vk27vce76c Taminya_ {
  peer: 2,
  transport: 110,
  transports: [
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { producer: 0 },
    { producer: 0 }, { producer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    ... 10 more items
  ]
}
request:getRouterRtpCapabilities
protoo Peer "close" event [peerId:marc1]
last Peer in the room left
protoo connection request [peerId:51......, address:/?roomId=6vk27vce76c&peerId=Taminya_&t=1646868788007, room:%s]
handleProtooConnection() | there is already a protoo Peer with same peerId, closing it [peerId:Taminya_]
protoo Peer "close" event [peerId:Taminya_]
existing room stat 6vk27vce76c Taminya_ {
  peer: 2,
  transport: 110,
  transports: [
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { producer: 0 },
    { producer: 0 }, { producer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    ... 10 more items
  ]
}
protoo connection request [peerId:51......, address:/?roomId=5b88wt1cw5l&peerId=marc1&t=1646861451977, room:%s]
existing room stat 5b88wt1cw5l marc1 {
  peer: 1,
  transport: 110,
  transports: [
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { producer: 0 },
    { producer: 0 }, { producer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    ... 10 more items
  ]
}
request:getRouterRtpCapabilities
request:getRouterRtpCapabilities
request:createWebRtcTransport
request:createWebRtcTransport
request:join
protoo connection request [peerId:51......, address:/?roomId=6vk27vce76c&peerId=Taminya_&t=1646868789950, room:%s]
handleProtooConnection() | there is already a protoo Peer with same peerId, closing it [peerId:Taminya_]
protoo Peer "close" event [peerId:Taminya_]
existing room stat 6vk27vce76c Taminya_ {
  peer: 2,
  transport: 110,
  transports: [
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { consumer: 0 },
    { consumer: 0 }, { consumer: 0 }, { consumer: 0 }, { producer: 0 },
    { producer: 0 }, { producer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 1 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 }, { producer: 0 }, { consumer: 0 },
    ... 10 more items
  ]
}
/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:161
                                        pReject(new Error('peer closed'));
                                                ^

Error: peer closed
    at Object.close (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:161:14)
    at Peer.close (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:103:9)
    at ConfRoom.handlePeerConnect (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:41:20)
    at WebSocketServer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/main.js:90:11)
    at WebSocketServer.emit (node:events:520:28)
    at WebSocketServer._onRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketServer.js:89:9)
    at WebSocketServer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketServer.js:37:50)
    at WebSocketServer.emit (node:events:520:28)
    at WebSocketServer.handleUpgrade (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketServer.js:224:14)
    at Server.emit (node:events:520:28)
[nodemon] app crashed - waiting for file changes before starting...

This crash occurs after 2h or 6h or 10h, it depends It's annoying because everything works fine but it always crashes after a few hours

The problem comes from protoo server or from meeting-mediasoup or from the mediasoup server?

lmangani commented 2 years ago

This seems a bad connection handler or a stack size overflow on the protoo side to me based on the error? You might want to check what's going on with the affected line/function to begin a theory as of the root cause:

/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:161

Madriix commented 2 years ago

https://github.com/versatica/protoo/blob/master/server/package.json#L25 It was automatically set to 1.0.34

I ran this to be in the same state as in 2020, the server has just started, have to wait about 6h or 12h to see if it crashes : debian@ns307:~/node/mediasoup/meething-mediasoup$ npm i websocket@1.0.33

Madriix commented 2 years ago

Between 1 a.m. and 9:45 a.m., it still hasn't crashed. On the other hand, I just saw an error, but it's cool because meeting-mediasoup didn't crash. I'm going to put it with pm2 and see if it crashes in 12h or 24h

    { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 }
  ]
}
request:getRouterRtpCapabilities
request:createWebRtcTransport
request:createWebRtcTransport
request:join
request:connectWebRtcTransport
_createConsumer() | failed:Error: request timeout
    at Timeout._onTimeout (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156:14)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7) {
  [stack]: 'Error: request timeout\n' +
    '    at Timeout._onTimeout (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156:14)\n' +
    '    at listOnTimeout (node:internal/timers:559:17)\n' +
    '    at processTimers (node:internal/timers:502:7)',
  [message]: 'request timeout'
}
protoo Peer "close" event [peerId:just]
protoo connection request [peerId:51.2...7, address:/?roomId=24hpbl828h8&peerId=just&t=1646987651919, room:%s]
existing room stat 24hpbl828h8 just {
  peer: 2,
  transport: 32,
  transports: [
    { producer: 1 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 },
    { producer: 0 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 },
    { producer: 1 }, { consumer: 0 },

Madriix commented 2 years ago

It's 5:00 p.m. and it still hasn't crashed, it was restarted this morning at 9:00 a.m. with pm2

That might be a good sign

I have to let it run all weekend, if it doesn't crash by Sunday evening then I had to go back to the previous version of "websocket".

If it does not crash, then on Monday I activate IPv6 again (dns, firewall, subdomain)

lmangani commented 2 years ago

Thanks for the update, keep us posted!

Madriix commented 2 years ago

The server crashed about 17 hours later :

/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:161
                    pReject(new Error('peer closed'));
                            ^

Error: peer closed
    at Object.close (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:161:14)
    at Peer.close (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:103:9)
    at ConfRoom.handlePeerConnect (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:41:20)
    at WebSocketServer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/main.js:90:11)
    at WebSocketServer.emit (node:events:520:28)
    at WebSocketServer._onRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketServer.js:89:9)
    at WebSocketServer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketServer.js:37:50)
    at WebSocketServer.emit (node:events:520:28)
    at WebSocketServer.handleUpgrade (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketServer.js:224:14)
    at Server.emit (node:events:520:28)

it's a shame that it crashes after a few hours and not immediately to better locate the problem

Madriix commented 2 years ago

I'm testing again with this version:

debian@ns307:~/node/mediasoup/meething-mediasoup$ npm i websocket@1.0.31

Madriix commented 2 years ago

It crashed tonight at 8:20 p.m :

request failed:Error: Peer already joined
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:96:37)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:59:12)
    at Peer.emit (node:events:520:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:520:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:520:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: Peer already joined\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:96:37)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:59:12)\n' +
    '    at Peer.emit (node:events:520:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:520:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:520:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'Peer already joined'
}
request failed:Error: Peer already joined
    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:96:37)
    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:59:12)
    at Peer.emit (node:events:520:28)
    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)
    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)
    at WebSocketTransport.emit (node:events:520:28)
    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)
    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)
    at WebSocketConnection.emit (node:events:520:28)
    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26) {
  [stack]: 'Error: Peer already joined\n' +
    '    at ConfRoom._handleProtooRequest (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:96:37)\n' +
    '    at Peer.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/lib/Room.js:59:12)\n' +
    '    at Peer.emit (node:events:520:28)\n' +
    '    at Peer._handleRequest (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:224:9)\n' +
    '    at WebSocketTransport.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:212:10)\n' +
    '    at WebSocketTransport.emit (node:events:520:28)\n' +
    '    at WebSocketTransport.safeEmit (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/EnhancedEventEmitter.js:18:9)\n' +
    '    at WebSocketConnection.<anonymous> (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/transports/WebSocketTransport.js:129:9)\n' +
    '    at WebSocketConnection.emit (node:events:520:28)\n' +
    '    at WebSocketConnection.processFrame (/home/debian/node/mediasoup/meething-mediasoup/node_modules/websocket/lib/WebSocketConnection.js:554:26)',
  [message]: 'Peer already joined'
}
/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156
                    pReject(new Error('request timeout'));
                            ^

Error: request timeout
    at Timeout._onTimeout (/home/debian/node/mediasoup/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156:14)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)

Madriix commented 2 years ago

I feel like there is a problem in meething-mediasoup/main.js and/or in meething-mediasoup/lib/Room.js

In fact maybe to solve it, the problem should already be solved first with mediasoup 3.9.9 ca it does not work with it, it displays an error related to "something must be accepted only one times", suddenly it accepts several and it immediately crashes the "npm start" launched.

Is it possible to tip to fix it right away, so that it works with mediasoup 3.9.9 ?

I also replaced this:

    if (lru.has(roomId)) {
        var room = lru.get(roomId);
        room.handlePeerConnect({
          peerId: peerId,
          protooWebSocketTransport: accept()
        });
            console.log("existing room stat", roomId, peerId, room.getStatus() );
    } else {
        var room = new ConfRoom(router);
        lru.set(roomId,room);
        room.handlePeerConnect({
          peerId: peerId,
          protooWebSocketTransport: accept()
        });
            console.log("new room stat", roomId, peerId, room.getStatus() );
    }

by

    if (lru.has(roomId)) {
        var room = lru.get(roomId);
            console.log("existing room stat", roomId, peerId, room.getStatus() );
    } else {
        var room = new ConfRoom(router);
        lru.set(roomId,room);
            console.log("new room stat", roomId, peerId, room.getStatus() );
    }

it was to see if it crashed, it didn't

Madriix commented 2 years ago

Maybe it's not Debian 11 the problem, maybe it's the new NodeJS 16. On the old dedicated server it was NodeJS 14. A few days ago I had a problem upgrading NodeJS 16.14, the old Hapi package didn't work anymore I had to update it to the latest version and now everything is fine. Let's imagine it's the same thing with stuff related to mediasoup?

I have no way to test meeting-mediasoup on an old nodejs 14. However I just thought that I had a dedicated server (another) there is NodeJS installed but no node application is active on this server, the version of node is v11.15.0, maybe I should put nodejs14 and test meeting-mediasoup but it requires a lot of work, I have to import the zip, set up the dns+domain+https, maybe I will have to test anyway

Madriix commented 2 years ago

Yesterday late afternoon on my second dedicated server (CentOS 7.9) I updated nodejs to 14 and also updated gcc as I had 4.8 it wouldn't compile mediasoup at all with the "npm i". So now I put a newer version of gcc and it (mediasoup) installed fine. I had to reboot this dedicated server after installation, a production server with over 100 sites.

Since last night meeting-mediasoup is running on my CentOS 7.9 + Node 14 + meeting-mediasoup default packages.json It's 1:00 p.m. and it hasn't crashed since.

Here is how the system was made:

CentOS server: node 14 + meeting-mediasoup
Debian 11 server: I changed nginx proxies instead of pointing to 127.0.0.1:2345 + 127.0.0.1:1234 - So instead it points to <CentOS Server>:2345 and <CentOS Server>:1234

it works well, I opened the ports between the two servers for both 2345 and 1234 and 20000:29999(tcp/udp) and without forgetting to put the ip of stun.l.google.com (108.177.15.127 ) in the permissions because otherwise the 2345 server didn't want to start 100%, it didn't run the console.log('Detected Server IP', pubIp);.

I'm testing until tomorrow morning 9:00 am, if it doesn't crash then that means it only crashes with the new Node 16.

On CentOS I would install Node 16 to see if it crashes between tomorrow 10:00 a.m. and 10:00 p.m.

lmangani commented 2 years ago

Thanks for keeping us updated with your results. It definitely seems like 14.x is the only stable version so far based on your testing. We'll add this to the readme.

Madriix commented 2 years ago

It's 8:00 p.m. It still hasn't crashed, it started around 9 p.m. yesterday. Finally I will not wait for tomorrow, I stop it now and I put nodejs 16 on CentOS and I turn it back on to see if it crashes in 2h or 4h or 12h max)

Madriix commented 2 years ago

npm ERR! gyp ERR! find Python checking if "python" can be used
npm ERR! gyp ERR! find Python - executable path is "/usr/bin/python"
npm ERR! gyp ERR! find Python - version is "2.7.5"
npm ERR! gyp ERR! find Python - version is 2.7.5 - should be >=3.6.0
npm ERR! gyp ERR! find Python - THIS VERSION OF PYTHON IS NOT SUPPORTED

My CentOS is too old, I wouldn't update Python. But the problem probably does not come from Debian11 but from Node 16.

Madriix commented 2 years ago

finally I installed python 3 and installed node 16 and typed the two "npm i", and meeting-mediasoup just crashed after 3 hours of operation:

request:getRouterRtpCapabilities
request:createWebRtcTransport
request:createWebRtcTransport
request:join
/home/node/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156
                                        pReject(new Error('request timeout'));
                                                ^

Error: request timeout
    at Timeout._onTimeout (/home/node/meething-mediasoup/node_modules/protoo-server/lib/Peer.js:156:14)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
[nodemon] app crashed - waiting for file changes before starting...

Are you going to make it compatible for node 16?

meething / meething-mediasoup

request timeout #13