render object and camera motion vectors

[sniklaus]

Huge thanks for sharing your stuff, this is great! I was just wondering how one would go about modifying RenderMotionVectors.cs to not only get object motion vectors but also camera motion vectors, do you have any pointers by chance? Thanks!

[Chuudy]

Huge thanks for sharing your stuff, this is great! I was just wondering how one would go about modifying RenderMotionVectors.cs to not only get object motion vectors but also camera motion vectors, do you have any pointers by chance? Thanks!

I would be also interested in this one.

[alelievr]

Hello,

With the current API, you can only use motion vectors in shaders (with the _CameraMotionVectorsTexture name). Modifying them is much harder though, that's why we're adding a new API in HDRP 12.0 that will allow to access the motion vectors buffer from C#.

I made a small example here: https://github.com/alelievr/HDRP-Custom-Passes/blob/2021.2/Assets/CustomPasses/CopyPass/CopyPass.cs#L79

[Chuudy]

Hi,

Thanks for your quick response. Unfortunately, I cannot load the project in Unity, as I receive an error on invalid dependencies. I believe it's due to some absolute paths in the project, as it states the path: 'C:\Users\AntoineLelievre\Github...'

With the current API, you can only use motion vectors in shaders (with the _CameraMotionVectorsTexture name).

Any guidelines on how to do it with shaders? I had an implementation that worked in the standard rendering pipeline in Unity 2019.4, but cannot make it work with HDRP in 2021.1.

[alelievr]

Yeah, there are a lot of examples in the code of HDRP like the TAA implementation here:

https://github.com/Unity-Technologies/Graphics/blob/608fe2362c4f3be978f86398b7db4859eb33c6e4/com.unity.render-pipelines.high-definition/Runtime/PostProcessing/Shaders/TemporalAntiAliasing.shader#L161

But basically, you need to use the DecodeMotionVector with the result of the sample of _CameraMotionVectorsTexture like so:

    float2 motionVector;
    DecodeMotionVector(LOAD_TEXTURE2D_X(_CameraMotionVectorsTexture, closest), motionVector);

DecodeMotionVector is in the include file BuiltinData.hlsl so make sure to add this include in your code: #include Packages/com.unity.render-pipelines.high-definition/Runtime/Material/Builtin/BuiltinData.hlsl

[Chuudy]

Thanks a lot, Antoine! I managed to render motion vectors to the render texture thanks to your help. I wonder is there any way to get camera motion vectors in the custom pass at the 'BeforeRendering' or 'AfterOpaqueDepthAndNormal' injection point? Perhaps it could be achieved when combined together with DrawRenderersCustomPass? I would be very grateful if you could tell me whether it's possible at all. My issue is that I need the motion vectors for the current frame during the rendering (including motion cause by animated objects and the camera itself). Before I was using a separate camera to render the motion vectors beforehand, but it required to go through the rendering process of the whole scene twice, and obviously wasn't very performant.

[alelievr]

I don't think that's possible without modifying HDRP. If you take a look at the pass graph of HDRP here, you'll see that the camera motion vector pass happens after the "After Opaque And Normals" pass:

May I ask for which kind of effect do you need them at this point in the pipeline?

[Chuudy]

You're right, it happens after the second injection point. That's why I hoped that maybe coupling it with some DrawRenderersCustomPass will force the engine to render out the motion vectors earlier.

May I ask for which kind of effect do you need them at this point in the pipeline?

Sure, I am not planning to do any post-process effect, I need the motion vectors for a current frame to steer the custom algorithm for NVidia VRS.

[alelievr]

FYI, the access to the camera motion vector buffer just got added in HDRP 12.0: https://docs.unity3d.com/Packages/com.unity.render-pipelines.high-definition@12.0/api/UnityEngine.Rendering.HighDefinition.CustomPassContext.html#UnityEngine_Rendering_HighDefinition_CustomPassContext_cameraMotionVectorsBuffer