search

pedroslopez / whatsapp-web.js

A WhatsApp client library for NodeJS that connects through the WhatsApp Web browser app
https://wwebjs.dev
Apache License 2.0
15.03k stars 3.57k forks source link

Puppeteer disconnects from the page when sending big files using MessageMedia.fromFilePath() (50MB+) #2117

Open gkp1 opened 1 year ago

gkp1 commented 1 year ago

Is there an existing issue for this?

Describe the bug

How is the bug happening?

// LATEST UPDATE AND TL;DR: Generating a big file using MessageMedia.fromfilepath(path) generates the MessageMedia (suspiciously) quickly with no errors, but when you try to send this MessageMedia, the puppeteer Page Closed disconnection occurs. Browser still running, can send messages manually through the webpage, but a full process restart is required to make wawebjs puppeteer work again. The crash starts occurring at around 50 or 60MB file size, anything higher than that crashes.

Generating custom MB size files for test: https://fastest.fish/test-files (generate a 90MB file and test)

Error 1: BIG FILES:

(whatsapp-web file size limit is 100MB but files above 50MB or around 80MB+ cause crashes)

Every time we try to send a file with around 70 ~ 90MB or more, for example a file.7z format, nodejs throws a string limit error. Error: Cannot create a string longer than 0x1fffffe8 characters originates from MessageMedia.fromFilePath(path); . This string error is not always present but the second one always happens: Then puppeteer disconnects from the running wawebjs chrome browser with the message Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. However, the chrome headless=false page does not close or crash and I can use whatsapp normally on it. It is not possible to close puppage or pupbrowser, also not possible to client.destroy() or cilent.initialize(). Full process restart required to close the browser and reopen

Error 2: MEDIUM FILES:

When sending files a little smaller like 49MB, there's no string limit error, and SOMETIMES it works and sends the file normally, but MOST TIMES (more than 50%), the whatsapp session silently breaks with the message {"name":"ProtocolError","originalMessage":"","message":"Protocol error (Runtime.callFunctionOn): Target closed."} and the session no longer works until the bot is restarted, the error repeats itself if I try to send any message after it occurred. But with headless=false you can see that the browser did not crash. Something with puppeteer or the browser broke, idk what. The bot session does not work anymore until you restart your bot, but I can still manually use the wa-web session normally through the chrome instance. Restarting makes it work again. New cool github text editor feature returns table formatting just by typing a slash /, so I made this: File size Works? Error Description
<= 40 MB Yes -
50 MB Sometimes 50% of the time it works and 50% breaks the session
80 MB No Target closed error
99 MB No Target closed + 0x1fffffe8 String limit

200% Relevant error logs:

Details: Click here to Open error logs

``` Error sending media, trying to send as document... Success sending as document! Error sending media, trying to send as document... Error: Cannot create a string longer than 0x1fffffe8 characters at Object.slice (node:buffer:648:37) at Buffer.toString (node:buffer:818:14) at Object.readFileSync (node:fs:504:41) at Function.fromFilePath (C:\node_modules\whatsapp-web.js\src\structures\MessageMedia.js:49:28) at C:\src\services\WbotServices\SendWhatsAppMedia.ts:27:33 at Generator.next () at fulfilled (C:\src\services\WbotServices\SendWhatsAppMedia.ts:5:58) { code: 'ERR_STRING_TOO_LONG' } Error: Cannot create a string longer than 0x1fffffe8 characters at Object.slice (node:buffer:648:37) at Buffer.toString (node:buffer:818:14) at Object.readFileSync (node:fs:504:41) at Function.fromFilePath (C:\node_modules\whatsapp-web.js\src\structures\MessageMedia.js:49:28) at C:\\src\services\WbotServices\SendWhatsAppMedia.ts:27:33 at Generator.next () at fulfilled (C:\\src\services\WbotServices\SendWhatsAppMedia.ts:5:58) { code: 'ERR_STRING_TOO_LONG' } Errors above sending media normally and sending media as document... WARN [23:09:49.541] (19560): message: "ERR_SENDING_WAPP_MSG" statusCode: 400 {"name":"ProtocolError","originalMessage":"","message":"Protocol error (Runtime.callFunctionOn): Target closed."} Error sending media, trying to send as document... ProtocolError: Protocol error (Runtime.callFunctionOn): Target closed. at C:\\node_modules\puppeteer\src\common\Connection.ts:304:16 at new Promise () at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:300:12) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) { originalMessage: '' } Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) Errors above sending media normally and sending media as document... WARN [23:10:29.723] (19560): Could not mark messages as read. Maybe whatsapp session disconnected? Err: WARN [23:10:29.919] (19560): message: "ERR_SENDING_WAPP_MSG" statusCode: 400 Error sending media, trying to send as document... Error sending media, trying to send as document... Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) Errors above sending media normally and sending media as document... Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) Errors above sending media normally and sending media as document... {} WARN [23:10:55.986] (19560): message: "ERR_SENDING_WAPP_MSG" statusCode: 400 WARN [23:10:55.993] (19560): Could not mark messages as read. Maybe whatsapp session disconnected? Err: {} WARN [23:10:59.713] (19560): message: "ERR_SENDING_WAPP_MSG" statusCode: 400 WARN [23:10:59.717] (19560): Could not mark messages as read. Maybe whatsapp session disconnected? Err: [INFO] 23:11:04 Restarting: \WbotServices\SendWhatsAppMedia.ts has been modified INFO [23:11:11.424] (17904): Servidor iniciado na porta: 8080 INFO [23:11:14.531] (17904): Client Connected INFO [23:11:28.522] (17904): Session: Normal AUTHENTICATED INFO [23:11:28.620] (17904): Session: Normal READY {"name":"ProtocolError","originalMessage":"","message":"Protocol error (Runtime.callFunctionOn): Target closed."} Error sending media normally, trying to send as document... ProtocolError: Protocol error (Runtime.callFunctionOn): Target closed. at C:\\node_modules\puppeteer\src\common\Connection.ts:304:16 at new Promise () at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:300:12) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) { originalMessage: '' } Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) Errors above sending media normally and sending media as document... WARN [23:19:12.024] (17904): Could not mark messages as read. Maybe whatsapp session disconnected? Err: WARN [23:19:12.417] (17904): message: "ERR_SENDING_WAPP_MSG" statusCode: 400 {} WARN [23:19:26.218] (17904): message: "ERR_SENDING_WAPP_MSG" statusCode: 400 WARN [23:19:26.226] (17904): Could not mark messages as read. Maybe whatsapp session disconnected? Err: {} WARN [23:19:32.033] (17904): message: "ERR_SENDING_WAPP_MSG" statusCode: 400 WARN [23:19:32.038] (17904): Could not mark messages as read. Maybe whatsapp session disconnected? Err: [INFO] 23:19:39 Restarting: WbotServices\SendWhatsAppMedia.ts has been modified INFO [23:19:42.550] (20716): Servidor iniciado na porta: 8080 INFO [23:19:58.571] (20716): Session: Normal AUTHENTICATED INFO [23:19:58.665] (20716): Session: Normal READY {"name":"ProtocolError","originalMessage":"","message":"Protocol error (Runtime.callFunctionOn): Target closed."} Error sending media normally, trying to send as document... ProtocolError: Protocol error (Runtime.callFunctionOn): Target closed. at C:\\node_modules\puppeteer\src\common\Connection.ts:304:16 at new Promise () at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:300:12) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) { originalMessage: '' } Error: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. at CDPSession.send (C:\\node_modules\puppeteer\src\common\Connection.ts:286:9) at ExecutionContext._evaluateInternal (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:254:44) at ExecutionContext.evaluate (C:\\node_modules\puppeteer\src\common\ExecutionContext.ts:140:23) at DOMWorld.evaluate (C:\\node_modules\puppeteer\src\common\DOMWorld.ts:174:20) Errors above sending media normally and sending media as document... ```

Expected behavior

Expected it to send files up to 100MB as whatsapp web allows.

However, on files tested with 49MB it works 50% of the time and 50% of the time it silently breaks your session without closing the browser, so you need to manually restart.

On files tested with 80MB to 100MB or even 102MB which is past web.whatsapp file limit, nodejs will throw string limit error and the same puppeteer error will happen, session silently crashes but browser stays running, requiring full api restart.

Steps to Reproduce the Bug or Issue

Put a 50MB, 90Mb or something and 101MB+ file in your app's folder and try to send them

Full function below

Relevant Code

Full test function

(tries sendAsDocument:true if normal media-sending fails)

Details: Click here to see code

```ts const tryToSendMediaNormallyOrDocument = async (messageMedia: MessageMedia) => { try { const sentMsg = await client.sendMessage(message.from, messageMedia, { sendAudioAsVoice: true, sendMediaAsDocument: false, }); if (sentMsg) { console.log("Success sending file normally!"); await message.reply("Success sending file normally!"); return sentMsg; } } catch (err) { console.log(err); console.log("failed to send file normally! TRYING TO SEND AS DOCUMENT..."); try { const sentMsgAsDocument = await client.sendMessage(message.from, messageMedia, { sendAudioAsVoice: true, sendMediaAsDocument: true, }); if (sentMsgAsDocument ) { console.log("Success sending media as document!"); await message.reply("Success sending file as document! (2nd try)"); return sentMsgAsDocument ; } } catch (error) { console.log(error); console.log("failed to send media as document. both tries failed!"); await message.reply("Both sendMediaAsDocument TRUE and FALSE failed."); await message.reply(err); await message.reply(error); return false; } } }; if (message.body== "sf" || message.body == "sendfile") { const media = { path: ["./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe","./YOUR/FILE/PATHS/HERE.exe", "./PUBLIC/FILES/DOC.DOCX", "./DUCKVIDEO.mp4"] }; let msgMedia: MessageMedia; media.path.every(async (path, index) => { msgMedia = await MessageMedia.fromFilePath(path); const sendingWorked = await tryToSendMediaNormallyOrDocument(msgMedia); if (sendingWorked) { console.log(`Sending message worked! ${index + 1}/${media.path.length}`); } else { console.log(`Sending message did not work! ${index + 1}/${media.path.length}`); message.reply(`Sending message did not work! ${index + 1}/${media.path.length}`); } await sleepRandom(1000, 2000); }); ```

Browser Type

Google Chrome

WhatsApp Account Type

Standard

Does your WhatsApp account have multidevice enabled?

Yes, I am using Multi Device

Environment

OS: [e.g. Mac, Windows, Linux, Docker + Ubuntu 18, etc] win11, will later test with ubuntu 22.x
Phone OS: [e.g. Android, iOS] android
whatsapp-web.js version [e.g. 1.2.3] 1.19.5
WhatsApp Web version [run await client.getWWebVersion()]: 2.2314.7
Node.js Version [e.g. 1.2.3] v19.0.0

Additional context

No response

amosayomide05 commented 1 year ago

It can only send 50mb document or video

I am affected by that bug too

matricce commented 1 year ago

Try this: https://github.com/pedroslopez/whatsapp-web.js/compare/main...matricce:whatsapp-web.js:fix/sendBigFiles

amosayomide05 commented 1 year ago

Try this: https://github.com/pedroslopez/whatsapp-web.js/compare/main...matricce:whatsapp-web.js:fix/sendBigFiles

Please what do it do

Can it send big files?

matricce commented 1 year ago

Try this: main...matricce:whatsapp-web.js:fix/sendBigFiles

Please what do it do

Can it send big files?

maybe.. fix: 'Protocol error (Runtime.callFunctionOn): Target closed.' when sending files larger than 80MB

matricce commented 1 year ago

Are you using the setRequestInterception method of Puppeteer (https://pptr.dev/api/puppeteer.page.setrequestinterception) in any part of the project?

amosayomide05 commented 1 year ago

Are you using the setRequestInterception method of Puppeteer (https://pptr.dev/api/puppeteer.page.setrequestinterception) in any part of the project?

Nope

matricce commented 1 year ago

Are you using the setRequestInterception method of Puppeteer (https://pptr.dev/api/puppeteer.page.setrequestinterception) in any part of the project?

Nope

I had the same problem of sending files larger than 50MB. The reason was that in my fork of the library, I was using the setRequestInterception method with a value of true. However, the behavior was a little different, and the process became "stuck". If I tried to send a file manually through WhatsApp Web, it kept loading infinitely.

gkp1 commented 1 year ago

Are you using the setRequestInterception method of Puppeteer (https://pptr.dev/api/puppeteer.page.setrequestinterception) in any part of the project?

Nope. I still haven't tried your fix. Will test soon. Thanks

EdenGottlieb commented 1 year ago

@gkp1 Did you manage to solve the issue reliably? Thanks!

amosayomide05 commented 1 year ago

@gkp1 Did you manage to solve the issue reliably? Thanks!

Nope

pierrecorsini commented 1 year ago

Try this: main...matricce:whatsapp-web.js:fix/sendBigFiles

This fixed the issue for me !

pierrecorsini commented 1 year ago

@matricce Thanks for the commit ! But sadly I dont see a PR with this commit ;)

gkp1 commented 1 year ago

Update: I am unable to test MATRICCE proposed solution as I'm not sure where to add the changes in the most recent Client.js? 😢

What happens is puppeteer disconnects from the page when sending big files (90MB for example). Even though the browser keeps running. It thinks page is closed. It is still possible to manually send messages through the running chrome browser whatsapp web, but puppeteer is no longer connected to it.

It is not possible to client.destroy() or client.initialize() or close pupPage or pupBrowser after this, a full process restart is required.


const knownBigFilesPuppeteerDisconnectionErrors = /Most likely the page has been closed\.|Protocol error \(Runtime\.callFunctionOn\): Session closed\./;

process.on("unhandledRejection", async (reason: any) => {
  const reasonString = String(reason); // Convert reason to a string
  console.log(`[unhandledRejection] ${reasonString}`);
  if (knownBrowserCrashErrorsRegex.test(reasonString)) {
    // log.info(`Client crashed. Destroying...`);
    // await activeClient.destroy(); // awaits eternally
    //await activeClient.pupPage.close(); // eternal error loop spam (don't run this)
    //await activeClient.pupBrowser.close(); // does nothing
    console.log("Client crashed. Restarting...");

 //pm2 restart myBot

  }
});

@pedroslopez @matricce @PurpShell is it possible to fix this issue on latest Client.js commit? I can test it for you. But sadly I have no idea how to implement a fix like matricce did. 😿

Atm the lib is limited to only sending ~40MB files

Whatsapp web 2.2328.5 Lib version pedroslopez/whatsapp-web.js#main (github)

tofers commented 1 year ago

https://github.com/pedroslopez/whatsapp-web.js/pull/2129 Update the puppeteer. It works for me and large files no longer cause a crash

matricce commented 1 year ago

2129 Update the puppeteer. It works for me and large files no longer cause a crash

I tried it here, but even doing tests with different versions of whatsapp, I couldn't make it work..

lib updated with main branch, chrome updated, node v18, puppeteer v18.. v20.. but I still couldn't send an pdf with ~80MBs

I don't know how you got it just by updating puppeteer and node.. :crying_cat_face:

gkp1 commented 1 year ago

2129 Update the puppeteer. It works for me and large files no longer cause a crash

2129 Update the puppeteer. It works for me and large files no longer cause a crash

I tried it here, but even doing tests with different versions of whatsapp, I couldn't make it work..

lib updated with main branch, chrome updated, node v18, puppeteer v18.. v20.. but I still couldn't send an pdf with ~80MBs

I don't know how you got it just by updating puppeteer and node.. 😿

@tofers Same thing, using OVERRIDES to replace puppeteer@13 with latest puppeteer@20.8.1, same error. I'm on Node 18.16.0 and using chrome .exe to run

Not sure if it's because it automatically uses puppeteer-core instead of puppeteer. Not sure if it's intended or if i should 'force' it to use 'puppeteer' instead of 'puppeteer-core'. But all I did was update Puppeteer 13 to 20.

Tried using new headless config " headless: 'new' " which v20 recommended, didn't impact anything.

FULL ERROR MESSAGE WITH PUPP@20.8.1

```shell sending file of size 90MB... 0|StickersBot | TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed 0|StickersBot | at CallbackRegistry.clear (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:153:36) 0|StickersBot | at CDPSessionImpl._onClosed (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:468:70) 0|StickersBot | at Connection._Connection_onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:338:17) 0|StickersBot | at WebSocket. (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\NodeWebSocketTransport.js:67:30) 0|StickersBot | at callListener (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:290:14) 0|StickersBot | at WebSocket.onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:220:9) 0|StickersBot | at WebSocket.emit (node:events:513:28) 0|StickersBot | at WebSocket.emitClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:258:10) 0|StickersBot | at Socket.socketOnClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:1264:15) 0|StickersBot | at Socket.emit (node:events:513:28) 0|StickersBot | at TCP. (node:net:322:12) 0|StickersBot | failed to send file normally! TRYING TO SEND AS DOCUMENT... 0|StickersBot | TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. 0|StickersBot | at CDPSessionImpl.send (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:429:35) 0|StickersBot | at ExecutionContext._ExecutionContext_evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:244:46) 0|StickersBot | at async ExecutionContext.evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:145:16) 0|StickersBot | at async Client.sendMessage (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\github.com+pedroslopez+whatsapp-web.js@8fc6e6098c88e7280074e4a9bdaabedc0646589e\node_modules\whatsapp-web.js\src\Client.js:815:28) 0|StickersBot | failed to send media as document. both tries failed! 0|StickersBot | TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed 0|StickersBot | at CallbackRegistry.clear (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:153:36) 0|StickersBot | at CDPSessionImpl._onClosed (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:468:70) 0|StickersBot | at Connection._Connection_onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:338:17) 0|StickersBot | at WebSocket. (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\NodeWebSocketTransport.js:67:30) 0|StickersBot | at callListener (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:290:14) 0|StickersBot | at WebSocket.onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:220:9) 0|StickersBot | at WebSocket.emit (node:events:513:28) 0|StickersBot | at WebSocket.emitClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:258:10) 0|StickersBot | at Socket.socketOnClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:1264:15) 0|StickersBot | at Socket.emit (node:events:513:28) 0|StickersBot | at TCP. (node:net:322:12) 0|StickersBot | TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. 0|StickersBot | at CDPSessionImpl.send (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:429:35) 0|StickersBot | at ExecutionContext._ExecutionContext_evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:244:46) 0|StickersBot | at async ExecutionContext.evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:145:16) 0|StickersBot | at async Client.sendMessage (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\github.com+pedroslopez+whatsapp-web.js@8fc6e6098c88e7280074e4a9bdaabedc0646589e\node_modules\whatsapp-web.js\src\Client.js:815:28) 0|StickersBot | [10:49:21.161] ERROR (16408): [unhandledRejection] TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed 0|StickersBot | [10:49:21.162] ERROR (16408): [unhandledRejection] TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed. ```

If anyone wants to test, this is how you do it.

Put this overrides in package.json, delete package-lock or pnpm-lock and node_modules and run npm i or pnpm i:

if using NPM:

```json { [...] "dependencies": { .... "whatsapp-web.js": "github:pedroslopez/whatsapp-web.js#main" }, "devDependencies": { .... }, "overrides": { "puppeteer": "latest" } } ```

if using PNPM:

```json { [...] "dependencies": { .... "whatsapp-web.js": "github:pedroslopez/whatsapp-web.js#main" }, "devDependencies": { .... }, "pnpm": { "overrides": { "puppeteer": "latest" } } } ```

for overriding only the specific whatsapp-web.js puppeteer instead of any puppeteer from all dependencies in the project, use: (probably not needed)

"overrides": {
"whatsapp-web.js":{
      "puppeteer": "latest"
}
    }
tofers commented 1 year ago

@matricce @gkp1

I have also tested it on clients.

It sends mp4 videos up to 64 MB. And 100 MB files (I didn't test more than that due to server load).

How do you send the files?

gkp1 commented 1 year ago

@tofers I try to send both normally and as document, same happens with any file format, using MessageMedia.fromFilePath(). The issue is probably with fromFilePath()? It succesfully sends anything up to around 50MB, supports videos and audios whatever, if video format is not supported, it sends the video as file normally, I can send .bin files of 20MB, 40MB, but 50MB or anything higher crashes

const sentMsg = await client.sendMessage(msg.from, messageMedia, {
        sendAudioAsVoice: true,
        sendMediaAsDocument: false,
      });
const sentMsgAsDocument = await client.sendMessage(msg.from, messageMedia, {
        sendAudioAsVoice: true,
        sendMediaAsDocument: true,
      });
tofers commented 1 year ago

And if you remove these options ? 

const sentMsg = await client.sendMessage(msg.from, messageMedia);

I don't use fromFilePath, as all files come as a link (e.g. link to s3 storage).

I download the file and send it to you already 

let fileBase64 = await axios.get(filePath, {responseType: 'arraybuffer'})
let returnedB64 = fileBase64.data.toString('base64')
let attachmentData = new MessageMedia(mime_type, returnedB64, filename);
gkp1 commented 1 year ago

@tofers Same thing removing the {options}. The crash isn't caused directly by fromFilePath(), but it crashes when sending the MessageMedia generated by fromFilePath(). I will rename the issue to clarify this 

// send a 90MB file
log.info(`Creating MessageMedia.fromFilePath size ${arg}...`);
    const msgMedia = await MessageMedia.fromFilePath(`./storage/${arg}.bin`); // --> generates the MessageMedia in 100ms
    log.info(`sending file of size ${arg}...`);
    tryToSendMediaNormallyOrDocument(client, message, msgMedia); // crashes

 [15:26:46.297] INFO (15456): Creating MessageMedia.fromFilePath size 90MB...
[15:26:46.415] INFO (15456): sending file of size 90MB...
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  |     at CallbackRegistry.clear (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:153:36)
0|StickersBot  |     at CDPSessionImpl._onClosed (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:468:70)
0|StickersBot  |     at Connection._Connection_onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:338:17)
0|StickersBot  |     at WebSocket.<anonymous> (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\NodeWebSocketTransport.js:67:30)
0|StickersBot  |     at callListener (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:290:14)
0|StickersBot  |     at WebSocket.onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:220:9)
0|StickersBot  |     at WebSocket.emit (node:events:513:28)
0|StickersBot  |     at WebSocket.emitClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:258:10)
0|StickersBot  |     at Socket.socketOnClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:1264:15)
0|StickersBot  |     at Socket.emit (node:events:513:28)
0|StickersBot  |     at TCP.<anonymous> (node:net:322:12)
0|StickersBot  | failed to send file normally! TRYING TO SEND AS DOCUMENT...
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
0|StickersBot  |     at CDPSessionImpl.send (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:429:35)
0|StickersBot  |     at ExecutionContext._ExecutionContext_evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:244:46)
0|StickersBot  |     at async ExecutionContext.evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:145:16)
0|StickersBot  |     at async Client.sendMessage (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\github.com+pedroslopez+whatsapp-web.js@8fc6e6098c88e7280074e4a9bdaabedc0646589e\node_modules\whatsapp-web.js\src\Client.js:815:28)
0|StickersBot  | failed to send media as document. both tries failed!
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  |     at CallbackRegistry.clear (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:153:36)
0|StickersBot  |     at CDPSessionImpl._onClosed (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:468:70)
0|StickersBot  |     at Connection._Connection_onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:338:17)
0|StickersBot  |     at WebSocket.<anonymous> (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\NodeWebSocketTransport.js:67:30)
0|StickersBot  |     at callListener (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:290:14)
0|StickersBot  |     at WebSocket.onClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\event-target.js:220:9)
0|StickersBot  |     at WebSocket.emit (node:events:513:28)
0|StickersBot  |     at WebSocket.emitClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:258:10)
0|StickersBot  |     at Socket.socketOnClose (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\ws@8.13.0\node_modules\ws\lib\websocket.js:1264:15)
0|StickersBot  |     at Socket.emit (node:events:513:28)
0|StickersBot  |     at TCP.<anonymous> (node:net:322:12)
0|StickersBot  | TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
0|StickersBot  |     at CDPSessionImpl.send (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\Connection.js:429:35)
0|StickersBot  |     at ExecutionContext._ExecutionContext_evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:244:46)
0|StickersBot  |     at async ExecutionContext.evaluate (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\puppeteer-core@20.8.1\node_modules\puppeteer-core\lib\cjs\puppeteer\common\ExecutionContext.js:145:16)
0|StickersBot  |     at async Client.sendMessage (C:\Users\Raposa\Desktop\Code\Bot_Stickers\node_modules\.pnpm\github.com+pedroslopez+whatsapp-web.js@8fc6e6098c88e7280074e4a9bdaabedc0646589e\node_modules\whatsapp-web.js\src\Client.js:815:28)
0|StickersBot  | [15:26:47.793] ERROR (15456): [unhandledRejection] TargetCloseError: Protocol error (Runtime.callFunctionOn): Target closed
0|StickersBot  | [15:26:47.794] ERROR (15456): [unhandledRejection] TargetCloseError: Protocol error (Runtime.callFunctionOn): Session closed. Most likely the page has been closed.
0|StickersBot  | [15:26:47.794] INFO (15456): Client crashed. Restarting...
0|StickersBot  | [15:26:47.794] INFO (15456): Sleeping 3000ms...
PM2            | Stopping app:StickersBot id:0
PM2            | App [StickersBot:0] exited with code [1] via signa
tofers commented 1 year ago

Can you give me the file, I'll try it

gkp1 commented 1 year ago

Can you give me the file, I'll try it

yep it crashes with any file extension so just specify 90MB file from this generate testfiles page https://fastest.fish/test-files Custom file size download

opssemnik commented 1 year ago

Try this: main...matricce:whatsapp-web.js:fix/sendBigFiles

this worked! However only for files up to 100MB - which is why i believe some folks still see issues (i assume because you just split it in the middle, creating 2x50MB chunks)

However wa web now supports 1gb files as attachments, would it be possible for you to update so it splits every 50mb?

opssemnik commented 1 year ago

Update: I am unable to test MATRICCE proposed solution as I'm not sure where to add the changes in the most recent Client.js? 😢

What happens is puppeteer disconnects from the page when sending big files (90MB for example). Even though the browser keeps running. It thinks page is closed. It is still possible to manually send messages through the running chrome browser whatsapp web, but puppeteer is no longer connected to it.

It is not possible to client.destroy() or client.initialize() or close pupPage or pupBrowser after this, a full process restart is required.

const knownBigFilesPuppeteerDisconnectionErrors = /Most likely the page has been closed\.|Protocol error \(Runtime\.callFunctionOn\): Session closed\./;

process.on("unhandledRejection", async (reason: any) => {
  const reasonString = String(reason); // Convert reason to a string
  console.log(`[unhandledRejection] ${reasonString}`);
  if (knownBrowserCrashErrorsRegex.test(reasonString)) {
    // log.info(`Client crashed. Destroying...`);
    // await activeClient.destroy(); // awaits eternally
    //await activeClient.pupPage.close(); // eternal error loop spam (don't run this)
    //await activeClient.pupBrowser.close(); // does nothing
    console.log("Client crashed. Restarting...");

 //pm2 restart myBot

  }
});

@pedroslopez @matricce @PurpShell is it possible to fix this issue on latest Client.js commit? I can test it for you. But sadly I have no idea how to implement a fix like matricce did. 😿

Atm the lib is limited to only sending ~40MB files

Whatsapp web 2.2328.5 Lib version pedroslopez/whatsapp-web.js#main (github)

You can just open his PR to see the entire sendMessage method https://github.com/pedroslopez/whatsapp-web.js/blob/cfacfbe7cc1957e452fc5d6dd93244589dc66353/src/Client.js#L635C1-L717C6

PwccaCode commented 10 months ago

Has this issue been fixed?