Layers created after the renderpass are not drawn (gizmos not rendering when renderpass is used)

[MAG-AdrianMeredith]

Description

When activating renderpass on a camera, the gizmo is no longer visible.

Steps to Reproduce

1. go to https://playcanvas.vercel.app/#/gizmos/transform-translate example

2. add the following code to enable renderpass

   const currentOptions = {
       camera: camera.camera, // camera used to render those passes
       samples: 0, // number of samples for multi-sampling
       sceneColorMap: true, // true if the scene color should be captured
       bloomEnabled: true,
   
       // disabled TAA as it currently does not handle dynamic objects
       prepassEnabled: false,
       taaEnabled: false
   };
   // Use a render pass camera frame, which is a render pass that implements typical rendering of a camera.
   // Internally this sets up additional passes it needs, based on the options passed to it.
   const renderPassCamera = new pc.RenderPassCameraFrame(app, currentOptions);
   
   // and set up these rendering passes to be used by the camera, instead of its default rendering
   camera.camera.renderPasses = [renderPassCamera];

3. gizmo is no longer visible

[mvaligursky]

It works if you add this render pass setup code to the camera after the gizmoLayer has been added to the camera. I'll try to add some warning when the layers are modified after the render pass has been set up or something.

[mvaligursky]

by the way, when you need to see what is happening with render passes (or even without them as they're used under the hood anyways), use

    pc.Tracing.set(pc.TRACEID_RENDER_PASS, log);
    pc.Tracing.set(pc.TRACEID_RENDER_PASS_DETAIL, log);

typically enable it for one frame, not every frame.

[MAG-AdrianMeredith]

Needing to run functions in a certain order is not ideal and error prone. Is there a layer added event or similar that could allow the render pass to sync up with the changes?

[mvaligursky]

Yeah it's on a longer term list for sure. But having a very quick look, it might not be too complicated to update that one pass internally.

[MAG-AdrianMeredith]

Yep there is I can see what needs to happen.

1. hook into the camera component layer change events
2. pass that to the renderpasses (add similar layer change events)

3. each render pass would then be able to optionally handle it if needed (similar to exising onEnable function etc)

   if its still open after the migration I'll take a look at it since I have a workaround for now

   Thanks

[MAG-AdrianMeredith]

Basically this means right now with renderpasses you cannot add/remove layers after initialisation which could be a big deal (we don't use this right now)

[mvaligursky]

It's just complicated in general way, I'll have to think about it, I'm not 100% sure what I want to do here. In the simple case, all layers from a camera are added. But in custom case, a render pass can use multiple cameras and add layers to them in custom order. For example for camera stacking, even though this will be doable by render pass merging I'm looking at adding soonish. So I'll have to figure out what flexibility I want to allow before I possibly implement it.

you cannot add/remove layers after initialisation which could be a big deal

you can simply re-create the render pass when the layers change:

    renderPassCamera.destroy();
    renderPassCamera = new pc.RenderPassCameraFrame(app, cameraEntity.camera, currentOptions);
    cameraEntity.camera.renderPasses = [renderPassCamera];

[MAG-AdrianMeredith]

Thanks, that sounds interesting. I fully accept we're on the bleeding edge here but it means we're flushing out a ton of good issues 😜

[tilemanrenovations]

I ran a vertical simulation and it did render. Weather it will in the right place at the right time is another question.

[mvaligursky]

yep keen on it and report all findings, all the feedback like this is fantastic!

[MAG-AdrianMeredith]

I'm guessing that other guy is a bot

[mvaligursky]

yeah I reported it as a spam .. if it keeps happening, we'll block it.

[tilemanrenovations]

I treat everyone the same.

[MAG-AdrianMeredith]

This issue is turning into a real deal breaker for my use case. It seems that any sort of changes to layers are inconsistently applied when renderpasses are applied for example setting a layer to be disabled/enabled has no effect with items in disabled layers still being rendered until the render pass is recreated. Theres lots of different places I'd want to modify layers and so this isn't a feasible solution.

Looks to be a straightforward fix, adding a check for ra.layer.enabled in RenderPassForward might do the trick. Though I am concerned that earlier on in "addLayer" its only adding layers if the layer is enabled which doesn't sound correct (i'd expect it to be added but no used). Not sure if thats causing any issue though, just something I noticed

[mvaligursky]

Yep, a good feedback and on my list to look into soonish. Thanks!

[MAG-AdrianMeredith]

I'm also struggling to get RenderTarget + RenderPass working, not sure what the issue is yet so a separate issue might be incoming

[mvaligursky]

I made some changes here: https://github.com/playcanvas/engine/pull/7001

Layers create after the render passes are still not handled, and at this stage I'm not sure I want to implement this, as this would add complexity and harder to find issues to code, which I'm hoping to avoid. Changes to layers (adding, changing order) affect more than just including them in forward render passes. It might remove a pass completely, change other flags on independent passes and similar.

I think for now I'll go with 'if you modify layers, recreate render passes' option for now.

[mvaligursky]

I'm also struggling to get RenderTarget + RenderPass working, not sure what the issue is yet so a separate issue might be incoming

I tried this few weeks ago by using render passes on a camera rendering to texture in this example and that worked. Please log any issue you find with this of course.

[MAG-AdrianMeredith]

New PR looks good, thanks! I have managed to get it rendering now (another race condition, I was adding the renderTarget to the camera after the render pass). But now it renders upsidedown

[mvaligursky]

Nice, I'll add a warning log for that as well I think. Regarding upside down, you might need something like this when creating render target: flipY: !this.app.graphicsDevice.isWebGPU or maybe just 'true'

[MAG-AdrianMeredith]

Tried both true and false and it seem to have no effect?!

[mvaligursky]

Tried both true and false and it seem to have no effect?!

I think the flip only works in forward passes, where we flip the projection matrix. Other passes do not handle this. If you have a custom shader reading from that texture, you can use getImageEffectUV or similar

[MAG-AdrianMeredith]

Its because theres another internal renderTarget called "SceneColor". when i set the flipY value in devtools it rights itself

pc.app.root.findComponents('camera')[1].renderPasses[0].rt.flipY = true

[MAG-AdrianMeredith]

Heres a fix, passing on the flipY value to the renderTarget inside of RenderPassCameraFrame when a renderTarget is passed

[mvaligursky]

Nice one, added it to the PR, thanks.

[mvaligursky]

I'll close this as I think it's in a good shape now. Please open separate issues as needed.