Open martonlederer opened 1 year ago
I've been thinking a lot about making this feature optional or specific to a type of messaging API. I've found that the best option we have is to implement this for long lived connection ports only.
The main reason for this is that the basic message flow is for smaller messages anyway and ports are for bigger data / longer data transfer time. It is also much easier to implement this with ports and I'm afraid the service worker could shut down after some time if we did this with the normal chrome.runtime.sendMessage
flow.
Running into this problem now, another solution might be - although prob a different RFC - but using Transferables with postMessage like over here https://groups.google.com/a/chromium.org/g/chromium-extensions/c/UyNHEHQKlJA/m/it_eC58DAwAJ
How do you envision this feature/change to look/work like?
The new system would automatically split the message data into pieces that wouldn't reach the size limitation of the browser messaging API. Each piece would then be sent as an
ArrayBuffer
to the recipient (ArrayBuffer
s can only be sent asnumber[]
) where it would be stored till it receives all chunks of the data. After that the recipient would reassemble it using the split data and extra information submitted with the chunk. After the reconstruction of the message, the chunks need to be deleted.There would be 3 different types of chunks:
Init chunks
The first chunk to send, it tells the recipient to start collecting chunks for this
chunkCollectionId
.Data chunks
Data chunks are the simplest chunks sent right after the init chunk has been sent.
End chunks
The last chunk to send, it tells the recipient to end the collection of the chunks for this
chunkCollectionId
.Chunked messaging flow process
Here is an sketch of how this would work between a content-script and a background service worker.
What is the purpose of this change/feature? Why?
In my testings, the maximum allowed size of messages is
67108862 bytes
or~67 Mbs
. This means that if you're trying to send a larger message or bytes of a bigger file, you can't use the browser's messaging API. The solution to this is chunked messaging, where developers don't need to worry about limits, because if they're handling larger data, the messaging library automatically splits their data into chunks, sends it to the message target and reassembles it on the other side. It would be also nice to allow developers to optionally customize max chunk sizes or otherwise use a fixed limit.Personally, we deal with large data transmissions in the ArConnect extension, sometimes even gigabytes, so we need a permanent solution to send these datas.
Example implementations
A primitive and basic implementation of this can be found at the ArConnect crypto wallet repo:
Code of Conduct