Failing to Receive large files via web app

[donpui]

Winden app fails (actually crash on receive side) while sending large files (500 MB, 1 GB or more) depending on user PC, Mobile device, browsers. There is no consistency of file size which fails. Currently we are limiting to 200 MB of files in Web App. Destiny Desktop/Mobile applications don't have this issue.

Original issue: https://github.com/LeastAuthority/MyFileTransfer/issues/25 Internal bug: #OP 238

There were already few attempts to fix this memory leakage in w-w wasm: https://github.com/LeastAuthority/wormhole-william/pull/76 and web app.

Minimal client for experimentation: https://github.com/LeastAuthority/wormhole-william-minimal-client

Few more tasks, ideas to try out:

• [ ] Try out with different websocket package size. Currently 16kb, may be should use maximum 64 kb
• [ ] Try out without using worker: https://github.com/LeastAuthority/winden/pull/93
• [ ] Experiment more with WritableStreamDefaultWriter to drop many running Promises: https://github.com/LeastAuthority/winden/blob/main/client/src/app/wormhole/client_worker.ts#L309-L313
• [ ] Search for a way to back-pressure on receiver side websocket messages
• [ ] Add bufferedAmount for sender side to (requires changes on nhooyr/websocket), however this doesn't solve main receiver side issue.

[donpui]

Performed some testing, as there were some refactoring, memory leak fixes.

Large File Transfer Test Cases - 2022-12-23.ods

Mostly fails receiving files on Mobile devices. Desktop is quite stable with 1 GB file transfer (send/receive). Even managed to send 5GB files.

Overall issue is on receiver side, when there is lack of memory to store all received data till full file received.

Also our current file size limitation on winden is incorrect setup. We should limit not only on sender side, but also on receiver, when file offer is presented.

[donpui]

@JustusFT @vu3rdd added few tasks to try out. Please put more, if something pops up.

May be we need to rethink whole process of receiving data, for example: may first get all bytes and push to temp file on system and only later on decrypt and generate downloadable file?

Also if we have mostly issues with mobile browsers, maybe we can separate limitation for desktop and mobile browser. For mobile experience, we could promote installable apps.

[donpui]

As issue still remains on Receiver side, which very much depends on Receiver computer/phone resources and browsers memory management.

Possible short term solutions:

• promote installable applications in Winden.app (https://github.com/LeastAuthority/winden/issues/88);
• remove file size check for sender on upload page, but add notification, that large files might fail on receiver side if web app is used. Promote installable app for receiver (Notify sender);
• add large file check on receiver when file offer is received:
  • if file is large then 100/200 mb. suggest to use install apps Desktop/Browsers (this could be depended from which platform he comes) and repeat upload process again, as code is already invalid;
  • notify, that download might fail. Failure most likely will happen for mobile browsers. While desktop browsers might succeed with 1GB transfers;

cc: @JustusFT @wuan