WebGLWorkerRenderer

[trusktr]

This is a feature idea.

The concept is simple: load the whole Three.js library inside a worker, make a scene as normal, then instead of rendering to a WebGLRenderer, one just renders to a WebGLWorkerRenderer.

This renderer will do the usual stuff that WebGLRenderer does: calculate world matrices and other CPU-blocking stuff, then send over a binary representation of the scene graph (SharedArrayBuffer) to the main thread. Maybe JSON would work too, especially for big scenes where stringify is small compared to the otherwise large amount of main-thread calculations.

On the main thread, a runner is responsible for querying the worker for data every frame, and on the main thread it will use something similar to a WebGLRenderer but with the computation parts removed, and it will render using the DOM canvas.

Of course, this is a simplified description to get thoughts going...

[mrdoob]

Have you experimented with Workers?

[Mugen87]

What about OffscreenCanvas? The section "Asynchronous display of frames produced by an OffscreenCanvas" describes an interesting approach. You render everything in a worker and commit frames to the canvas element in the main thread.

This approach is also mentioned here: https://www.youtube.com/watch?v=wkDd-x0EkFU&feature=youtu.be&t=1246

Later in the talk you can see an example with three.js 😁

[kaisalmen]

WorkerSupport in LoaderSupport provides helpful util functions to get most code into a worker easily (e.g. complete three.js). See this snippet especially line 150: https://github.com/mrdoob/three.js/blob/dev/examples/webgl_loader_obj2_meshspray.html#L117-L173 data.input can be a transferable. This is what OBJLoader2 does. In scopeBuilderFunc the worker messages are returned including the Transferables (various buffers here). All means are there to get OffscreenCanvas running easily. So far, WorkerSupport is only used by OBJLoader2, MeshSpray= worker triangle generator, and soon PCDLoader, but its use is already not constrained to loaders. Sounds like a nice experiment for the upcoming days... 😄

Btw, SharedArrayBuffers were deactivated per default in Firefox and Chrome because of Meltdown: https://blog.mozilla.org/security/2018/01/03/mitigations-landing-new-class-timing-attack/ https://www.chromium.org/Home/chromium-security/ssca

[matthias-w]

OffscreenCanvas could also be a nice approach for generating dynamic textures using CanvasRenderingContext2D. AFAIK, currently only WebGLRenderingContext is available in browsers.

BTW I already use WebWorkers for generating all my THREE.js geometries including the generation of an Octree for THREE.BufferGeometry. I also use it for Constructive Solid Geometry which would otherwise block my rendering loop. However, you have to be careful when transferring recursive algorithms to WebWorkers. I used this CSG lib which works good in browsers' main thread. When I used it in a WebWorker I got many stackoverflow exceptions when applied to more complex geometries. The algorithms in the lib are implemented recursively. After some web search I figured out that maximum stack size in WebWorkers is much lower than in main thread. I changed the algorithm implementation to an iterative approach. After that, the CSG lib works within WebWorkers. This way, CSG can be done also for complex geometries taking a larger amount of time (in my app sometimes 2s-10s) without blocking your UI.

[donmccurdy]

I would suggest that this be explored with some examples before any new WebGLWorkerRenderer API is created... I'm not sure the arrangement of keeping THREE.Scene content on the main thread and ferrying it through Transferrables to a Renderer in the WW is the right abstraction.

There is an OffscreenCanvas example now (#13253), IMO that is enough for general purposes.

[backspaces]

Our team is beginning to use workers and decided that building a library/tool around the main-worker threads was not as important as refactoring our code into worker compliant modules.

So basically we looked at which of our modules could easily be refactored into a worker-able part and the main/DOM part.

So basically, as long as Three can define which modules are worker-able, then all is fine.

[trusktr]

Looks like simply using OffscreenCanvas isn't that simple. It is only simple if all your Three.js code is inside of the worker.

---

But it is complicated, if for example, you have a React application that touches DOM elements (f.e. UI on top of a WebGL scene), and various components (including 3rd-party components) can create THREE.Object3D instances (THREE.Mesh, etc) and need to pass them into the scene for rendering.

If the OffscreenCanvas and WebGLRenderer are in a worker, now just imagine the userland complexity around having to pass objects into the worker (serialized) in order to create what's needed inside the work for rendering.

To make this super simple for users, we'd need to make some sort of interface components (f.e. THREE.MeshUI vs THREE.MeshWorker where one lives in the UI thread, and the other in the worker thread, and updates to the UI instance propagate into the Worker instance.

Without such an interface, it is impractical to use OffscreenCanvas in applications that use frameworks like React, Angular, Vue, etc, unless the worker entirely contains Three, and the user makes some sort of interface for communicating all scene changes from the UI thread to the WOrker thread (basically the idea of the previous paragraph). It requires a LOT of effort.

No web-based 3D engine has ever done this before, that I know of.

[mrdoob]

I don't think that's something three.js should be responsible for.

[backspaces]

@trusktr: I agree.

For our effort, the Model/View split helped, the Model runs in the Worker. The postMessage(data) is either an Object of TypedArrays (x,y,theta + user variables) or the View is run in the Worker too, and the data is an ImageBitMap. Both used as Transferable Objects.

We're currently exploring Workers with 2d canvas but also have a Three View which we think would work the same way: the data would be the typed arrays for a buffer geometry or the rendered ImageBitMap. Oddly enough, with some very simple experiments, the ImageBitMap is the most performant!

@mrdoob: I agree if core Three.js just uses the canvas and no direct access with the DOM. Or at least if a Rollup of subset of Three would provide a no-DOM Three. I'm guessing it will Just Work as is, or with minor changes.

[backspaces]

@mrdoob https://rawgit.com/mrdoob/three.js/dev/examples/#webgl_worker_offscreencanvas works, I think this means we're Go! Sorry to have missed this.

[looeee]

@backspaces support for offscreencanvas is limited:

https://caniuse.com/#feat=offscreencanvas

[Bug-Reaper]

So I can make 3D objects and stuff in a web worker but not anything that uses image textures as it requires dom access to a canvas? Would be great to have a way to load a new scene without dropping fps to 0 for a few seconds while everything is loaded in.

[munrocket]

OffscreenCanvas is limited, but there is interesting way with webgl-workers how to fix it in firefox. So probably it is a good suggestion. Here some benchmark of this solution.