ReadableStream not transferable \w postMessage

[jimmywarting]

Tried starting a Worker thread and transfer a RedableStream but it is not transferable

minimal repro.

var rs = new ReadableStream()
var mc = new MessageChannel()
mc.port1.postMessage(rs, [rs])

[rracariu]

According to MDN, these stream types should be transferable:

ReadableStream WritableStream TransformStream

[josephrocca]

Oof - spent a while preparing to remove a server's bottleneck with web workers under the assumption that Deno would have support for this.

In case it's useful to know: For my use case I have a server handling a lot of requests (several thousand per minute). Generating the (long-running) response stream is somewhat compute intensive and involves writing several hundred (sometimes less, sometimes much more) tiny chunks for each response stream, so I want to offload all that to workers, and have the main thread only receive client requests, pass those requests to a worker, get the resulting stream back, and return that readable stream to the client in a Response within Deno.serve.

@bartlomieju Is there any chance of this happening some time soon? Or is there any other scaling paradigm that you'd direct devs towards for a streaming use case like this?

[josephrocca]

I realised that it's possible to kind of crudely "polyfill" this with SharedArrayBuffers - so that's what I ended up doing for now. Unfortunately I didn't write it as a general-purpose polyfill, so it's not in a sharable state at all, but figured I'd mention it for others hitting this thread from a web/issue search.

[mmastrac]

@josephrocca We're currently looking at a way to automatically shard Deno.serve. Transferring streams may be a difficult thing for us to do at this time.

[josephrocca]

@mmastrac Oh, that's exciting! Will this basically be a behind-the-scenes load balancer process + multiple distinct Deno processes? If so, this is less ideal for my current use cases, but would certainly still be handy on future projects.

The reason I like using web workers where possible is because it makes it much easier to performantly coordinate global state/metric tracking/etc. Having the main thread be for "serving and coordinating", and the worker threads be for "processing" means that with only a small extra mental overhead (which is alleviated a lot by Comlink - and there are very exciting ECMAScript-level things in the works here), I can get a single Deno process to handle as much as 100x more traffic in some cases.

Multi-process is inevitable as you keep scaling, but there's a lot of good DX to harvest before that I think. It's just a lot slower and more cumbersome when you don't have easy and efficient communication (via transferrables + shared memory) between nodes. As much as I'd like to not have ~any communication between nodes (per Best Practices) - the real world often says "no".

[josephrocca]

Tangentially, I'm warming up to SharedArrayBuffer as a mechanism for extremely efficient response serving/streaming, but Deno.serve doesn't really seem to support this paradigm because you need to return a Response. Ideally I'd return a "virtual Response" that is e.g. specified by a SharedArrayBuffer for the response body, plus another shared buffer containing a currentLength (since we're streaming response bytes into the buffer) and a doneFlag, or something very vaguely like that.

The idea is to obviate the need for any main-thread work after returning the SharedArrayBuffer in Deno.serve and passing said SharedArrayBuffer (plus the currentLength/doneFlag/metadata SharedArrayBuffer) to worker. IIUC, this would make serving extremely scalable - basically linear with number of workers I'd imagine (but I could be completely wrong - maybe Rust-V8 boundary adds some performance limitations here). Currently I need to poll or Atomics.waitAsync the currentLength/doneFlag and make a lot of calls to writableStreamWriter.write on the main thread.

[bartlomieju]

@mmastrac Oh, that's exciting! Will this basically be a behind-the-scenes load balancer process + multiple distinct Deno processes? If so, this is less ideal for my current use cases, but would certainly still be handy on future projects.

No, it would be a single process with N threads (defaulting to no of cores on your machines) and each of them runs a separate instance of the program. You can still "talk" between the threads using BroadcastChannel.