question: custom pass for rendering video texture or spout stream (incoming render texture)

[sasa42]

Hi,

Thaks for the nice demos. Very useful.

Would it be possible to add an custom pass who allows to have a clean rendering of a video texture or incoming render texture stream (spout stream).

TAA and DOF should work in this example. For now TAA is smearing the the video or render texture stream or DOF is not working.

As an use case we have full ui simulations for e.g both of these screens of the car interior where you can interact with (touch display). These are streamed in to the material as render textures. Movie textures are another use case. https://beforesandafters.com/wp-content/uploads/2019/04/UNITY_RT_05-768x432.jpg

For now there is no solution that TAA and DOF is working and to display crisp content on both of the interior displays when camera is moving or you drive the car.

Thanks.

edit: Extendind this example

https://user-images.githubusercontent.com/6877923/99796085-29382480-2b2d-11eb-89b8-73c1cd16af48.png

by mapping the UI per RT Texture to one of the green motor blocks would be a good use case and would be working with you demo assets. Alternativ a video texure mapped on one of the green motor blocks.

For now when the ui is scrolling or something is moving inside the UI (rt texture or video texture in material) the TAA smears the motion or when you do an after postprocess dof is not working.

[alelievr]

Hello,

Indeed, I didn't look at this problem yet but I think it's possible to "remove" the TAA from the world space UI by doing this:

• Render the world space UI in a separate render texture with a custom pass
• Composite the camera color buffer and the UI render texture in a custom post-process with the injection point "Before Post Process"

The key thing here is to use the custom post-process "Before Post Process" injection point which will execute your code just after the TAA pass and before all the other post-process as shown in this diagram:

I'll probably add an exampling using this technique in the future but feel free to try this if it works for your case :)

[alelievr]

Here you go: https://github.com/alelievr/HDRP-Custom-Passes/blob/master/README.md#render-video-without-taa :)

[sasa42]

Hi. Thanks a lot. The quality from the content is very good now. Is it possible to get rid of the flickering borders?

The main view is always at the car displays, which are interactive, like here https://www.kreisbote.de/bilder/2021/03/16/90245361/25226620-das-bmw-idrive-bediensystem-im-cockpit-eines-bmw-ix-2uRcyY5af0ea.jpg and a flickering of the borders takes all user attention.

[alelievr]

Have you tried the fixDepthBufferJittering option in the render without TAA custom pass?

It re-renders all the objects in the depth buffer so it can be pretty expensive, but AFAIK it's the only wait to get rid of all flickering

[sasa42]

Yes. There is a little flicker at camera movement on the borders left. But i know. This is hard to eliminate. I also downgraded to 2020.3.x LTS and HDRP 10.5.0 and i am really happy that it even works there.

However. Because you drive the car, the camera has driver seat position and you interacting with these UI screens this i a visible artifact who attracts all user attention.

In BuiltIn RP i could elminate this and got smooth borders by rendering two cameras (first interior, second UI planes) and by adding two times CTAA with different settings (TAA Asset Store) to these. The screen content was not so crisp like your solution but the borders were fine.

[sasa42]

Thanks again for the custom pass NoTAA

Attached a screenshot to better visualize the real beauty.

upper screen: normal rendering of an injected spout stream -> TAA smears everything and fast moving content (last digits) are badly smeared and not readable at all because moving in sub second speed

screen below: Custom Pass, NoTAA rendering -> good readability from static and fast moving screen content

The moving time texture stream is sent in via spout from an external unity app via keijiro´s new DX12 Spout implementation

[alelievr]

FYI I discovered that with recent versions of HDRP, the object in this custom pass were rendered with jittered matrices, I made a fix here: https://github.com/Unity-Technologies/Graphics/pull/5088

[sasa42]

Thanks. I will test. A question in advance for upcomin DLSS integration in 2021.2.x.

Can i expect similar behaviour there or is DLSS not comparable with TAA at all.

Because this is our main workflow to stream up to 5 interactive UI´s to the car scene i try to understand this advance. But DLSS is pretty new for me.

[alelievr]

DLSS shouldn't change anything because it happens before TAA :)

[sasa42]

This is also a method to render moving content during path tracing! Here pathtracing is started with 4096 samples. Screenshot is taken at half time.

Very helpful.

upper plane, normal rendered spout input stream.is frozen when pathtracing started and not readable because moving time.

under plane, rendered per custom pass without TAA -> even during pathtracing the animated input stream is moving nicely (fast moving time counter here) and it is rendered sharp.