Add readAsSharedArrayBuffer to FileReader

[topicus]

To allow workers play with file contents currently you have three options (maybe more):

• Post a message to the worker with the File (structured cloning is costly).
• Read the file as an array buffer and transfer it to the worker context (it's no longer accesible from other contexts).
• Read the file as an array buffer and copy it's contents to a shared array buffer (still a copy is needed).

If you need to parallelize tasks over file contents (e.g. parsing a big csv file) there is no way to do it without perform a copy which is costly operation when you deal with large files.

It would be great to have a readAsSharedArrayBuffer to read the file straight in a shared array buffer and thus avoid the copy. It would be cool to allow file reading in parallel but that's another issue.

Is there any other way to perform this with the current specs that I'm missing?

[domenic]

This is the wrong level at which to solve this problem. We should go around duplicating every API on the platform that produces ArrayBuffers to create a second parallel version that creates a SharedArrayBuffer. If nothing else, the cost of keeping all these APIs in sync is very high.

Instead, what you need is a feature (probably an ECMAScript-level feature) for creating a SharedArrayBuffer from a non-shared ArrayBuffer, without memory copies. (Presumably this would detach the original ArrayBuffer.)

[domenic]

Also,

Post a message to the worker with the File (structured cloning is costly).

[citation needed]. It should be extremely cheap, especially for Files (which behind the scenes are basically a filename).

[topicus]

@domenic About File structured cloning I misinterpreted a benchmark, it's very cheap as you mentioned.

Regarding readAsSharedArrayBuffer maybe adding a param to readAsArrayBuffer can be an option:

fileReader.readAsArrayBuffer([blob[, shared]])

It would be backward compatible and just change the underlying class to be used.

Besides that I agree with you it would be great to allow create a SharedArrayBuffer from an ArrayBuffer. Maybe something like this https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer/transfer for SharedArrayBuffers.

SharedArrayBuffer.transfer([oldBuffer[, byteLength]]);

[jimmywarting]

do we need a way to do the opposite of something like SharedArrayBuffer.transfer then? Some API don't accept a sharedArrayBuffer like

new Blob([shared])
new File([shared], '')
crypto.getRandomValues(shared)
crypto.subtle.digest(opts, shared)
new Response(shared)
new Request('/', {method: 'post', body: shared})
new RTCPeerConnection().createDataChannel('label').send(shared) // not tested

There is no native api that i know of that accepts a SharedArrayBuffer except for postMessage the only thing you can mostly do is to read/write data to a shared buffer

[annevk]

What kind of behavior would you expect there? A copy at the IDL boundary? Why not copy it yourself into an ArrayBuffer? Not ergonomic enough?

[annevk]

Closing this in favor of #143.