local + global effect pipeline troubles

[braebo]

Hey there! First off -- this library is incredible and the work you all do is nothing short wizardry, so thank you!! 🙏

Recently, I've hit a roadblock trying to manage multiple layers in a particular way.

In this case, I have what I'm calling a local selection layer, which has 1 mesh (or more) assigned to its selection layer, and has its own effects passes. Then, I have a global selection layer that includes all meshes, including the objects in local. The idea is to apply local effects to just 1 (or more) object(s), then apply global effects on top to all objects.

Problem is -- I'm struggling to get the local layer with its applied effects included into the global pass. As it is currently, the local layer completely ignores any effects in the global layer even though it's selected. I think I vaguely understand why this is happening, but I'm not entirely sure how to fix my implementation.

I've put together this diagram that shows my current understanding of how my pipeline works, along with the (slightly modified) code:

Pipeline Example Diagram


Pipeline Code Example

```typescript const selection_local = new Selection([], 15) const selection_global = new Selection([], 16) const renderPass = new RenderPass(scene, camera) const savePass = new CopyPass() // Store the result of the local effects in a texture. const textureEffect = new TextureEffect({ blendFunction: BlendFunction.ALPHA, texture: savePass.texture, }) const blendPass = new EffectPass(camera, textureEffect) const composer = new EffectComposer(renderer, { frameBufferType: HalfFloatType, }) // Global Passes const clearPass_global = new ClearPass() const LambdaPass_global = new LambdaPass(() => camera.layers.set(selection_global.layer)) const effectPass_global = new EffectPass(camera, ...globalEffects) const convolutionPass_global = new EffectPass(camera, ...globalConvolutionEffects) // Local Passes const clearPass_local = new ClearPass() clearPass_local.overrideClearAlpha = 0.0 const lambdaPass_local = new LambdaPass(() => camera.layers.set(selection_local.layer)) const effectPass_local = new EffectPass(camera, ...localEffects) const convolutionPass_local = new EffectPass(camera, ...localConvolutionEffects) // Build the composer function buildPipeline() { // Render the local objects first. composer.addPass(clearPass_local) // Clear the working buffer. composer.addPass(lambdaPass_local) // Set the local layer mask. composer.addPass(renderPass) // Render the local scene. composer.addPass(effectPass_local) // Apply the local effects. for (const pass of convolutionPasses_local.values()) { composer.addPass(pass) // Apply any local Glitch / Pixel / Shockwave passes. } composer.addPass(savePass) // Save the local result to a texture. // composer.addPass(global.clearPass) // Clear the working buffer. // todo - has no effect? composer.addPass(lambdaPass_global) // Set the layer mask to select everything (global). composer.addPass(renderPass) // Render the global scene. composer.addPass(effectPass_global) // Apply the global effects. for (const pass of convolutionPasses_global.values()) { composer.addPass(pass) // Apply any global convolution effects. } composer.addPass(blendPass) // Blend the local and global passes. // todo - The savePass isn't being included in the global effects pass. // todo - How can we include it? } ```

If my understanding is correct -- what I want to do is somehow include the output buffer of the savePass into the input buffer of the global phase (sometime before adding the effectsPass_global)? If so, I'm not entirely sure how I should approach that.

A bit more context for the curious

I've been tasked with porting the postprocessing part of an "editor" I've built to this library (previously built using three's builtins and a lot of custom shader passes). The "editor" basically allows a user to prototype a scene with an arbitrary combination of effects passes on each layer (`global`/`local`), toggling them on and off and re-ordering them (that part is a bit more involved with this library I've discovered 😅). Over the past week, I've been reverse engineering and wrapping each effect one by one and it's been an enlightening experience! I've certainly been forced to learn a lot in a short period of time due to the lower level nature of this library compared to three, which has been refreshing for me. I've managed to get quite far in a week thanks to the abundance of information that's been shared in issues in this repo throughout the years 🙏. The examples were life savers (and are all broken now it seems, not sure whatsup with codesandbox but I was able to copy the code over locally to experiment with).

Anyways, thanks again for the amazing library! I'm really looking forward to working with it more (and contributing if there's any way I might be able to help), and I'm particularly exited for v7 🚀

Thanks in advance to anyone who may be able to provide some insight!

[braebo]

I think I figured it out!

I just needed to move the blend pass up the chain so that it blends in the save pass before the global effects are rendered 👍

[braebo]

Nvm -- jumped the gun there a bit. It looks like I lost the initial local effects from the save pass entirely. It appears they are being overwritten by the global pass.

[vanruesc]

Hi, thanks for the kind words!

How are you managing your object selections? If you're using the Selectionclass, are you using different render layers? The default layer is 10. See also Layers.

Oh, I missed the example :sweat_smile: I'll take a closer look when I get a chance.

[vanruesc]

Actually, have you tried disabling the internal ClearPass of the renderPass?

renderPass.clearPass.enabled = false

// todo - The savePass isn't being included in the global effects pass. // todo - How can we include it?

The blendPass receives the result of the global effect chain as input and blends that with the texture assigned to the TextureEffect.

[braebo]

Hey thanks for the advice! I seem to be getting the same result even if I set that to false though.

In this example, the cube is added to local and the background image is added to global. It seems the output of the save pass (local effects) isn't being processed by the global effects -- the cube is unaffected by the global pipe: and is instead blended at the end of the pipeline.

https://user-images.githubusercontent.com/55504972/227045455-fe4335c7-5495-412b-aa5e-20ec7007492a.mp4

If I'm understanding correctly -- what I want to do is include the output buffer of the save pass (containing the local render + effects result) in the input buffer of the global effect pass?

Would it be better if I put together an example on codesandbox / stackblitz?

[braebo]

Getting warmer!! I've got the local object + effects being run through the global effects by replicating the FeedbackEffect and FeedbackPass approach in the example you gave here (and just removing the Blur from it).

Problem now is that the local (cube) layer renders behind the global layer (bg image) presumably because local is rendered first. Is this because the local effects include a convolution effect (glitch)?

I'm going to dive into this example for some clues. I can't seem to be able to locate the issue it's from, but the example has 2 cubes with 1 always "on top" occluding the other regardless of position, and it seems you you were demonstrating how to control depth.

Fixed that too! I had disabled depthTest on the cube at some point...

The cube's glitch artifacts are no longer visible beyond the bounds of the object though 🤔

[braebo]

I'm going to close this for now since my new problem is unrelated. Thanks for the input!