option/API to automatically force preload all material uploads to GPU

[ngokevin]

Description:

A-Frame/three.js gets serious framedrops whenever we display or load a material for the first time. Even if an entity has a material instantiated, if the entity or its parent is non-visible, the renderer will hang if you flip them to visible. It seems the three.js WebGL renderer does not upload non-visible object's materials.

What we want is a way to preload all the materials up front on scene load so that the user does not experience frame drops during the experience. We can do this by flashing/drawing the materials on initialization, and then removing them. Currently, the base material component reuses all common materials in the material system, we can create a single three.js Object3D, and cycle through all of them to get them all pre-uploaded.

For the API, perhaps we can have a disablePreloadGPU property on the material that can opt-out of this. But what about the case of materials not created from the base material GPU? We could also have a public method, perhaps on the material system, that does this like el.systems.material.forceUpload(threejsmaterial).

Reproduce: https://aframe-360-gallery.glitch.me

Open Firefox Performance Tools, start recording, swap back and forth between images, frame drops when image is first loaded, smooth on second load.

[donmccurdy]

I assume you've seen it already, but ~50% of this thread is relevant: https://github.com/mrdoob/three.js/issues/11746. Specifically, the parts about incremental GPU uploads.

My preference, if we have developer cycles to spend on it, would be to try to get that implemented instead. In some common-enough cases (like scenes with a bunch of sequential 360º images) holding all textures in GPU memory probably isn't ideal. But non-blocking texture upload is harder to do in WebGL1; it's possible it won't work well until we can take advantage of WebGL2... But in any case, I think that's the best long-term outcome.

If we do need to hack something within A-Frame in the meantime, let's try to keep that as API-neutral as possible. Three.js only uploads textures the first time they are rendered, as you say, but there are hacks to upload a particular texture without needing to actually render the model.

For the API, perhaps we can have a disablePreloadGPU property on the material that can opt-out of this.

Many (most?) textures will also come from 3D models, in which case the material component isn't involved. We could make a loaders system shared by *-model components... maybe some other options?

But what about the case of materials not created from the base material GPU?

What do you mean?

We could also have a public method, perhaps on the material system, that does this like el.systems.material.forceUpload(threejsmaterial).

Yes, I think i'd be in favor of an opt-in solution rather than opt-out.

[ngokevin]

Yeah, I'd prefer to have a solution in the meantime over the year it takes for the WebGL2 solutions to come to fruition. Since it affects probably all scenes.

Perhaps a global scene flag that detects all materials/textures being created (extend the loaders) and flashes them on start. Doing that I get smooth performance after first load.

[ngokevin]

But what about the case of materials not created from the base material GPU?

I meant material component*, asking about e.g., materials created via model loaders or custom.

[ngokevin]

Some methods in THREE.WebGLRenderer and THREE.WebGLTextures, we can try to use to manually invoke the GL calls to upload textures.