depthPass no longer works with postProcessing in r167

[Spiri0]

Description

I updated my app to three.js r167. I think the new structure of three.js is very good. This now gives the project the possibility to cleanly separate webGPU and webGL. Except for one small thing in the passNode, everything runs fine with r167. The function "getTextureDepthNode()" in the passNode was thrown out. This is now probably done together with the "getTextureNode()" function and you have to specify the name 'depth' for the depthTexture as ​​I understand it. Unfortunately, this doesn't work as hoped. But that doesn't look like a big problem.

Reproduction steps

I made a very simple codePen. I think the code speaks for itself. I made the CodePen example so that it runs without depthTexture so that you see an executable reference.

As soon as you insert these two lines for the depthTexture into the shader header, it no longer works.

viewportDepthTexture: texture_depth_2d,
viewportDepthSampler: sampler,

Code

//here just the init and renderloop. To emphasize the essential. The complete code with the three.js setup is on codePen

async function init() {

  const material = new THREE.MeshBasicNodeMaterial();
  material.colorNode = vec4(0, 0.25, 0.75, 1); 
  const geometry = new THREE.BoxGeometry( 1, 1, 1 );
  const box = new THREE.Mesh( geometry, material );
  scene.add( box ); 

  postProcessing = new THREE.PostProcessing( renderer );
  const scenePass = pass( scene, camera );
  const scenePassColor = scenePass.getTextureNode();
  const sceneDepthPass = depthPass( scene, camera );
  const sceneDepthPassColor = sceneDepthPass.getTextureNode('depth');

  //if i include this lines in the shader in line 58 the code no longer work.
  //viewportDepthTexture: texture_depth_2d,
  //viewportDepthSampler: sampler, 

  const shaderParams = {
    uv: viewportTopLeft,
    viewportTexture: scenePassColor,
    viewportSampler: scenePassColor,

  }

  const fragmentShader = wgslFn(`
    fn main_fragment(
      uv: vec2<f32>,
      viewportTexture: texture_2d<f32>,
      viewportSampler: sampler,
      viewportDepthTexture: texture_depth_2d,
      viewportDepthSampler: sampler, 
    ) -> vec4<f32> {

      return textureSample(viewportTexture, viewportSampler, uv);
      //return textureSample(viewportDepthTexture, viewportDepthSampler, uv);
    }
  `);

  postProcessing.outputNode = fragmentShader(shaderParams);  
}

function render() {

  postProcessing.render();

  requestAnimationFrame( render );
}

Live example

https://codepen.io/Spiri0/pen/MWMpzGd

Screenshots

No response

Version

167

Device

No response

Browser

No response

OS

No response

[Mugen87]

This was changed in #28833.

pass.getTextureDepthNode() should now be pass.getTextureNode( 'depth' ). I'm not sure why your raw WGSL code does still not work though, see https://jsfiddle.net/or97ms3t/1/.

The value of the returned texture node is definitely a depth texture...

[Spiri0]

then I at least understood correctly with the

getTextureNode( 'depth' )

I get a long error message in the console on the desktop. The fiddle you created only works if I remove the two lines with the depthTexture from the shader like in codePen and in my app.

Do you also see the long error message in your console? I can paste these here if you wish

[Mugen87]

Yes, I've seen it but I don't understand the root cause yet.

[Spiri0]

To test it, I imported the DepthTexture from three, the way it was in r166.

import { DepthTexture } from 'three';

I suspect that this is still the same as in r166. But I didn't check that. The error message is exactly the same. I admit that I still don't understand the error message. It doesn't seem to be the DepthTexture itself that is the problem. The error message indicates a problem with the bindings. But that's just speculation on my part.

update: The depthTexture itself is in fact not the cause. I tested this with

import { DepthTexture } from "https://cdn.jsdelivr.net/npm/three@0.167.1/src/textures/DepthTexture.js/+esm";

//const depthTex = new THREE.DepthTexture(512, 512);
const depthTex = new DepthTexture(512, 512);

and

const shaderParams = {
  uv: viewportTopLeft,
  viewportTexture: scenePassColor,
  viewportSampler: scenePassColor,
  viewportDepthTexture: texture(depthTex), //sceneDepthPassColor,
  viewportDepthSampler: texture(depthTex), //sceneDepthPassColor,    
}

const fragmentShader = wgslFn(`
  fn main_fragment(
    uv: vec2<f32>,
    viewportTexture: texture_2d<f32>,
    viewportSampler: sampler,
    viewportDepthTexture: texture_depth_2d,
  ) -> vec4<f32> {

You can see it in the codePen example. The cube is shown, which would not be the case if the depthTexture caused an error. That explains your statement that the texture is definitely a depthTexture. And it confirms that both the DepthTexture from three.webgpu.js and the DepthTexture from src/textures/DepthTexture.js work with the texture node. So there is something wrong with the depthPass. Who made the changes? Maybe the one remember the crucial changes related to the depthPass and nodeObject. It seems to be a binding problem because it works with the texture node but not with the PassNode, which takes over the creation of the bindings in post-processing.

[Spiri0]

Here again the link with the updated example and added DepthTexture tests and comments. @sunag I hope this helps you. So far I haven't been able to figure out where things are going wrong with the bindings, which I suspect as the cause.

https://codepen.io/Spiri0/pen/MWMpzGd?editors=0010

[Spiri0]

I used the new function in r167.1 in PassNode to take a closer look into the textures. Top: output Bottom: depth

getTexture('output');
getTexture('depth');

The depthTexture was initialized in r166 in the constructor of the PassNode with:

this._depthTextureNode = nodeObject( new PassTextureNode( this, depthTexture ) );

In r167.1 the initialization of the depthTexture in the constructor is missing. This explains why when I try getTexture('depth') instead of getTextureNode('depth') I get the red error message in the console that width is missing. The long yellow error message about the sampler in the console is probably a subsequent error because the depthTexture is not calculated at all. And therefore it is not present in the GPU or is not present correctly.

[Mugen87]

I don't think undefined dimension values are the root cause. The depth texture is created as before here:

https://github.com/mrdoob/three.js/blob/239b639fbb6af1330c137727c03b571044663b73/src/nodes/display/PassNode.js#L85-L92

If there would be an error in this part, evaluating depth information would be broken in general and certain post processing passes like DOF would not work.

Notice that the dimensions of a depth texture are configured automatically by the renderer here:

https://github.com/mrdoob/three.js/blob/239b639fbb6af1330c137727c03b571044663b73/src/renderers/common/Textures.js#L51-L59

If you set a breakpoint there, you should see the resize works correctly in my fiddle as well.

[Spiri0]

I admit I can't get any further at the moment. I will be patient until Sunag or someone else who understands this better than me finds the cause and can solve it. There is no example with the depthPass node in the examples. An example would certainly be very practical after the issue has been fixed. I had been looking for a way to get the depthTexture of the scene into a raw WGSL shader with the node system for about half a year (from time to time alongside my other projects). As you can see from the example I did, it was ultimately very easy with the node system if you know how. The depthPass with the getTextureDepthNode() was the solution. This means that in the example you can see the depthTexture without the issue if you comment the corresponding return in the shader. A very simple example but very helpful.

I use the depthTexture to calculate atmosphere. The terrain (r166) still looks quite boring here, but the atmosphere is exactly as it should be.

I have to look for a suitable server provider to make this online experienceable. This is only possible thanks to webgpu in three.js. Unfortunately this doesn't work on github. I can't upload 1.5 million maps and configure the server as necessary. I need a own server. In my opinion the name webgpu experimental can be removed from the description of three.js because it is no longer experimental. It's awesome ❤️

[Mugen87]

Is this the expected result? https://jsfiddle.net/rb5fu4gn/

The root cause is that post processing now supports multisampling (#28784). If you enable it via antialias: true, the depth texture gets multisampled as well. In this case, you must transform texture_depth_2d into texture_depth_multisampled_2d and process the texture differently.

Would this simplified setup work for you as well? https://jsfiddle.net/73pLgzhb/

[Spiri0]

Yes, that's exactly what looks right, just the depth output. This is what my codepen example in r166 looked like. I immediately looked in the code to see what you did differently. So I wouldn't even need texture_depth_multisampled_2d because you get the multisampled depth value directly into the shader as f32. That seems to me to be better than going through the textures in the shader and only taking the r channel from them anyway. With your variant of sending the depth value already sampled directly into the shader as f32, it looks more functional.

I would never have thought of doing that with f32 because I only know it as a uniform or attribute. The fact that I get direct access to the respective depth value of a texture or multi-sampled textures with f32 is new for me. I haven't seen that in the W3C description yet. Is this luxury part of the node system?

[Mugen87]

Not sure about this. Maybe related: https://github.com/gpuweb/gpuweb/issues/2094

[Mugen87]

In any event, I believe this issue can be closed. After all, it is not a bug or regression in the engine.

[Spiri0]

I'm not sure whether the example is too simple as an example for three.js. But in my opinion, this way of using a depthTexture in post-processing is already worth an example. I look at the examples a lot and find them very good as a help with WebGPU