Change SCREEN_TEXTURE behaviour to align with 2D: copy when an object uses it and add max copies.

[QbieShay]

Describe the project you are working on

Various VFX, 3D sprites, tech art.

Describe the problem or limitation you are having in your project

Screen texture is currently computed between the opaque and the transparent pass, making it impossible to refract on transparent objects. Use cases for this are for example a beach with foliage below the sea surface (seaweed), a VFX that has some distortion shockwave.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

SCREEN_TEXTURE is copied every time there is an object that uses screen texture, and at the same time there is a maximum cap per-viewport of the amount of times the screen texture can be copied. This would allow to have transparent objects refracted into object that use screen texture, and the copy cap the viewport settings (or project?) will be used to ensure that there can't be too huge fluctuations of fps when rendering different parts of the game.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

See above

If this enhancement will not be used often, can it be worked around with a few lines of script?

It changes the rendering pipeline

Is there a reason why this should be core and not an add-on in the asset library?

It changes the rendering pipeline

[Zireael07]

My workaround: use one of multiple "dither as fake transparency" shaders around. (Use case: am using SCREEN_TEXTURE for some super fakey motion blur)

[clayjohn]

We discussed this earlier, this would be a pretty complex change, but one idea for how this could work would be to maintain a list of alpha draw lists. The screen texture would be copied between each list and back to front front draw ordering would be maintained across lists. For example, if only one object uses SCREEN_TEXTURE then two alpha lists will be created, one with all objects behind the one with SCREEN_TEXTURE and one with the object using SCREEN_TEXTURE and all objects in front of it. I am not sure how performant an approach like that would be, we would definitely need to profile carefully.

[viktor-ferenczi]

Copying here clayjohn's ideas from our chat thread where this issue came up:

"Maybe we could add a texture hint to the screen texture that tells the renderer to recapture immediately before drawing this mesh."

"The challenge would be making it compatible with the Vulkan mobile renderer which requires you to set up all your render-pass information before you start rendering. So basically you need to know when and where you change render targets before rendering starts."

[viktor-ferenczi]

Re-capturing the SCREEN_TEXTURE is relevant only if transparent meshes are behind each other. Also, the game developer will know in which cases this is relevant for the gameplay, even if it happens technically. I agree giving control over the re-capture on a per mesh basis, so the game developer can mark such meshes before each frame. This would also avoid having too many draw lists.

[jitspoe]

At the very least, I would love to be able to get the SCREEN_TEXTURE including alpha surfaces. Sometimes I want to do a localized warp effect or something, and that causes all the transparent particles to disappear.