Depth of Field: incorrect masking with transparent materials

davcri commented 5 months ago

Description of the bug

Rendered image isn't correct when using DoF with transparent materials.

To reproduce

https://codesandbox.io/p/sandbox/react-postprocessing-dof-isssue-forked-2zcyyg

Screenshots

UPDATE: the original post contained only the previous sandbox. I added these screenshots afterwards.

Codesandbox link: Current rendering with 6.34.3. Cube with transparent MeshStandardMaterial and envmap reflections

Desired effect

Codesandbox link: custom postprocessing to illustrate the desired rendering with transparent MeshStandardMaterial, sharp edges and blurred background. The code does not take depth into account and it probably has wrong light calculations.

Workarounds

depthWrite=false: fixes the background but blurs also the edges of the transparent object.

vanruesc commented 5 months ago

The cube is marked as transparent but its opacity is 1. To me this looks like it's working as intended: the reflective cube is in focus and the background is blurred.

davcri commented 5 months ago

The example is a bit misleading since I'm using a MeshPhysicalMaterial with transmission (transparency=true has no effect).

However I get the same result by using a MeshStandardMaterial with transparency and opacity < 1.0 (https://codesandbox.io/p/sandbox/react-postprocessing-dof-isssue-601-meshstandardmaterial-p4grw6?file=%2Fsrc%2FApp.js%3A65%2C74)

The main issue is that the object writes to the depth buffer and this avoids blurring the background, even if it should be.

Another workaround would be increasing roughness on the MeshPhysicalMaterial, however I think this isn't still a realistic effect (since reflections would be affected, in a similar way to depthWrite=false).

vanruesc commented 5 months ago

This won't work since the DepthOfFieldEffect has no influence on the textures that are used for transmission/reflection sampling. Does backgroundBlurriness address your use case?

davcri commented 5 months ago

It would work for envmap backgrounds but it wouldn't work with proper meshes (which unfortunately is what i have in the project I was working on while I found out the bug).

I took this image from a Unity thread, hopefully this clarifies what I'd like to achieve:

PS: thanks again for all the replies @vanruesc , sorry for being pedantic but I didn't find anything online regarding ThreeJS and I'd like to find a fix for this (probably not so common) problem.

vanruesc commented 5 months ago

The main issue is that the object writes to the depth buffer and this avoids blurring the background, even if it should be.

The DepthOfFieldEffect blurs objects based on depth and that's working as intended in this case. We can't see the actual background through the transmissive object - that's effectively just its texture.

The deeper problem with transmission is that the renderer renders the scene to a transmission texture in an internal process that can't be modified. If this was customizable, we might be able to apply DoF to the transmission texture that is later read by the transmissive object. We'd then have to render DoF again for the fully rendered scene, but we'd end up blurring the transmissive object twice...

I don't know if there's a solution to this.

davcri commented 5 months ago

Yeah you're right about MeshPhysicalMaterial, however the problem affects also other materials with transparency, correct?

I don't know if there's a solution to this.

Me neither, but I'd lke to investigate about the required changes in ThreeJS and Postprocessing in order to have a proper support for DoF and transparent objects out of the box. Do you know anyone in the community that could give some hint?

In the meantime I updated the first comment with:

a simplified explanation of the problem/workarounds
a codesanbox with a simulation of the desired effect (naive approach that doesn't take depth into account)

In the simulation I did something like this:

composer.addPass(new RenderPass(sceneWithOpaqueObjects, camera));
composer.addPass(new EffectPass(camera, new DepthOfFieldEffect(camera, {...})));
composer.addPass(new CopyPass(opaqueRenderTarget));
composer.addPass(new ClearPass(true, true));
composer.addPass(new RenderPass(sceneWithTransparentCube, camera));
composer.addPass(new CopyPass(transparentRenderTarget));
// render transparent texture on top of the opaque texture
composer.addPass(new EffectPass(camera, new CustomEffect(transparentRenderTarget.texture, opaqueRenderTarget.texture)));

vanruesc commented 5 months ago

however the problem affects also other materials with transparency, correct?

Yes, it applies to all transparent meshes. However, transmission is a different problem compared to true transparency. And transparency comes with different blend modes: alpha-blend, alpha-test and alpha-hash.

Alpha-Blend has the same issue as you've encountered with transmission.
Alpha-test is not affected because it's just a cutout based on alpha.
Alpha-hash with TAA might just work, but I haven't tried that since we don't have a good TAAEffect yet.

Do you know anyone in the community that could give some hint?

No, and I don't think anyone would have a definitive answer. Probably relevant: https://forum.unity.com/threads/transparent-rendering-with-dof.784823/

hybridherbst commented 5 months ago

proper support for DoF and transparent objects out of the box

That one's a hard problem as there's only one layer of depth buffer (in typical implementations) and it's not clear where the DoF on transparent objects should "go to". E.g. at opacity 0.1 you likely want blurred background (doesn't write depth), and at 0.9 you likely want no blur (writes depth) for a focus point on the transparent object.

Transmission has the same issue in the other direction (it writes depth, and at transmission = 0.1 you d expect it to behave like any other opaque object, but at transmission=1.0 you'd expect behaviour closer to transparent).

There are workarounds as mentioned above:

expensively compositing multiple layers (what movie production does)
various dithering approaches ("alphaHash" in three.js) that can work ok for full-resolution depth buffers depending on the DoF implementation

vanruesc commented 4 months ago

I feel like this is one of those issues that are bound to stay open forever since there's no clear solution. Would it be ok if I convert it into a discussion instead?

davcri commented 4 months ago

@vanruesc ok for me

pmndrs / postprocessing