WebGLRenderer: Allow for binding, rendering into mipmap storage of textures

[gkjohnson]

Description

Rendering custom mipmaps can be valuable for a number of use cases for post processing, stylization, etc but it's not something that three.js supports currently. Use cases include:

• Mipmap generation for non-color data maps such as normal, depth maps (useful for postprocessing, etc).
• Manual generation of per-layer mipmaps in array textures, 3d textures.
• Manual generation of mipmaps to account for wrapping constants
• etc

I think there are a few concept disconnects currently. One is that "generateMipmaps" indicates that both mipmap storage should be generated and the mip chain should be generated. When generating custom mipmaps though these concepts should be separate. Ie you may want an option that says "generateMipmapStorage" and "generateMipmapContents". Or a setting that enumerates the three options. Another is that you cannot currently render into just any textures storage.

cc @CodyJasonBennett

Solution

These are some solutions that come to mind - there are no doubt others. I can't say these are optimal or align with what's possible in WebGPU but I'll list them here to start the discussion:

Generating Mipmap Storage w/o Contents The generateMipmaps setting could be changed to take three options so attaching storage does not implicitly mean generating content: NO_MIPMAPS (current false), MIPMAP_STORAGE, or MIPMAP_CONTENTS (current true).

Rendering to Mipmaps Currently setRenderTarget supports taking a activeMipmapLevel but as far as I can tell this will only work if the user has specified textures in the texture.mipmaps array, is a 3d texture, or cube map. The active mipmap level could also apply to the automatically-generated mipmap storage using the framebufferTexture2D.

Writing to Regular Texture Mipmaps The above solutions only really apply to RenderTargets but generating custom mipmaps for regular textures, normals maps, data textures, etc are all relevant. A simple solution would be to enable setting a regular non-rendertarget texture as a depth buffer-less renderable target.

Alternatives

**Generating Mipmap Storage w/o Contents** To do this currently you can create a render target, initialize it with `generateMipmaps = true`, and then disable it to ensure the storage is available. This however still incurs the overhead of generating mipmaps on creation: ```js const map = new THREE.WebGLRenderTarget( 32, 32, { generateMipmaps: true } ); renderer.initRenderTarget( map ); map.texture.generateMipmaps = false; ``` **Rendering to Mipmaps / Writing to Regular Texture Mipmaps** Using `copyTextureToTexture`, custom mipmaps can be generated with render targets and then copied into the appropriate mipmap level. The additions in #29769 allow for copying any existing mip map data, as well. This solutions incurs unneeded overhead copying and an additional render target, however.

Additional Context

WebGPU does not support automatic generation of mipmaps: https://github.com/gpuweb/gpuweb/issues/386 The answer to this [stackoverflow question](https://stackoverflow.com/questions/79109103/how-to-copy-specific-mip-map-level-from-a-source-texture-to-a-specific-mip-map-l/79134417#79134417) shows that it's possible to render into a mipmap storage while sampling from the immediate parent mip by setting the `TEXTURE_MAX_LEVEL`, `TEXTURE_BASE_LEVEL`, `TEXTURE_MAX_LOD`. Setting these can probably be left to the user.

[CodyJasonBennett]

Perhaps it's best to accept a number for pure storage rather than overloading an array of texture.mipmaps like webgl_materials_cubemap_render_to_mipmaps. We don't implement transform feedback, but then that would have a similar desire for buffer attributes, which are then only GPU-facing. They have a similar overload in WebGL (or mappedAtCreation: false for WebGPU) for when you don't want to allocate CPU memory just to specify a size, in our case for many mips.

https://github.com/mrdoob/three.js/blob/841ca14e89f3ec925e071a321958e49a883343c0/examples/webgl_materials_cubemap_render_to_mipmaps.html#L117-L118

Implementation-wise, it would be nice to support TEXTURE_MAX_LEVEL, TEXTURE_BASE_LEVEL, TEXTURE_MAX_LOD as texture/render target properties to avoid ping-ponging (https://jsfiddle.net/cbenn/g617aq93) and MRT if able for this API. I've recently used them in conjunction for SSILVB/XeGTAO, although I ran into a plethora of platform issues with NPOT via ANGLE/DirectX (https://issues.chromium.org/issues/40877668). These are texture APIs that remain useful with WebGPU code, and I believe all of the early SPIR-V and now WGSL examples for mipmap generation use TEXTURE_BASE_LEVEL, although you can port SPD (MIT) with some difficulty.