Support transferable objects with web workers

[bitspittle]

Web workers are now out and they're powerful, but if you have to send HUGE amounts of data between the barrier, it can be extremely slow (think the image data of a 2000x4000 png). Can we update the generated worker class somehow to allow passing in raw transferable inputs / outputs?

See also: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Transferable_objects#supported_objects

[bitspittle]

Potentially a way this could work is if there were more WorkerFactory base classes to choose from, like

class ImageWorkerFactory : WorkerFactory.RawIO<ArrayBuffer, ArrayBuffer> {
   ...
}
// Or WorkerFactory.RawI<ArrayBuffer, ComplextType>
// Or WorkerFactory.RowO<ComplexType, ArrayBuffer>

where the workers generated in that case wouldn't try to serialize the value, they would just send it directly.

[bitspittle]

The previous comment isn't quite right, because in theory, users can pass in MULTIPLE transferable objects. After sleeping on it, I think the way forward is to add an optional "extra" parameter which is a map of string to Any value pairs:

worker.postInput(input, mapOf("imageData" to imageData))

at which point, input would still get serialized into a string, but the extra parameters would not. Instead of internally sending a string to the worker (current behavior), we'd send a json struct like this (some pseudo code here):

// worker implementation
assertTransferableObjects(imageData)
_worker.postMessage({
  _input: Json.encodeToString(input),
  imageData: imageData
}, [imageData])

At that point, it will be up to the worker code to cast between Any and the property type of the value.

And then the WorkerFactory implementation will look like this:

override fun onInput(extras: Map<String, Any>, postOutput: (Output, Map<String, Any>) -> Unit) = ...
// or
// class WorkerInputContext(extras: Map<String, Any>)
//  override fun WorkerInputContext.onInput(postOutput: (Output, Map<String, Any>) -> Unit) = ...

---

This definitely would complicate the worker factory interface and would not be backwards compatible. This seems like a shame because extra parameters are really a performance optimization feature that most users shouldn't need to worry about.

Because of that, I feel like we should introduce a new interface, maybe:

class ImageWorkerFactory : WorkerFactory.Extra<Input, Output> {
   ...
}

so if a user wanted to migrate their code for performance reasons, it would be an explicit opt-in step.

Meanwhile, the WorkerProcessor KSP code will need to be updated to look for WorkerFactory.Extra implementations in addition to WorkerFactory implementations, generating a slightly different worker implementation in that case.