How to implement PostProcessing like Spatial/Temporal/DecimationFilter in Unity

[Jerice]

https://github.com/IntelRealSense/librealsense/issues/1572

• Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • Consider checking out SDK examples.
  • Have you looked in our documentations?
  • Is you question a frequently asked one?
  • Try searching our GitHub Issues (open and closed) for a similar issue.

• All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)

---

Required Info
Camera Model	D415
Firmware Version	05.09.09.02
Operating System & Version	Win 10
Kernel Version (Linux Only)
Platform	PC
SDK Version	lastest
Language	unity(c#)

Issue Description

Hi, I want to implement the filter of the SDK in Unity. Is that possible? I have tried to add this filters with no success. Thanks!

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment] @Jerice Attached my way to implement new post processing block for Unity.

• First to implement the customized post processing inside C# by leveraging the class of "CustomProcessingBlock", and then compile this class as one of the Intel.RealSense.dll.
• Copy the newly compiled Intel.RealSense.dll to Unity to wrappers/unity/Assets/Plugin.Managed.
• Copy the Unity script calling the customized class implemented inside the C# to wrappers/unity/Assets/RealSenseSDK2.0/Scripts. Please download the package from the comment below.

[carypai]

Unity_PostProcessing_sample.zip

[Akthree]

Hey Guys, I also was looking forward to implement the post-processing filters inside my Unity Project. But i cant quite follow your instructions given.

First to implement the customized post processing inside C# by leveraging the class of "CustomProcessingBlock", and then compile this class as one of the Intel.RealSense.dll.

What do I need to put inside where? Can you please provide a little more details for this particular step? The second Question is: Do i just attach the CustomProcessorDemo.cs as Component on my GameObjects or is it a more global approach? I am very thankful for the support but i would also like to understand what i need to do :D

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment] @Akthree After unzip the package, follow the instruction below :

1. Copy the "CustomProcessor.cs" to /wrappers/csharp/Intel.RealSense (optional, you can add the existing item from anywhere in your system in step 2)
2. In visual stuido, add existing item (CustomProcessor.cs) from Intel.RealSense project, then build it. Two main dll files will be generated, realsense2.dll, Intel.RealSense.dll
3. Create "Plugin.Managed" folder under /wrappers/unity/Assets, then copy the "Intel.RealSense.dll" generated in step 2 to this folder.
4. Create "Plugins" folder under /wrappers/unity/Assets/RealSenseSKD2.0, then copy the "realsense2.dll" generated in step 2 to this folder.

For how to use in Unity, since I am not very familiar with it, I just modify the "Color" object inside StreamsTexture (in Asset/RealSenseSDK2.0/Scenes/RealSenseTextures) for testing. The way should be the first way you mentioned, to add the GameObject then attach the CustomerProcessorDemo as its component.

[Akthree]

@RealSense-Customer-Engineering
Thanks for the help! So i got it all working properly but i still aint getting anywhere. I attached the CustomerProcessorDemo-Script on the Color StreamsTexture just like you mentioned and its all working just fine but when i attach it to the Depth StreamsTexture it is somehow breaking the pipeline and there is no useful output.

I also digged a little deeper inside the c# wrapper examples and got a little bit of an insight on how to implement postprocessing. But just to be clear the CustomerProcessorDemo.cs file currently is not doing a lot exept breaking the depth channel. Can someone provide a little more information on how to expose the filters inside the CustomProcessor and how to manipulate the values compared to the C++ postProcessing example or the Realsense-Viewer? So if someone can tell me how to expose and apply the filters inside given Demo i would be glad to work on a better implementation for Unity.

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment] @Akthree The example I show you intended to let the script inside Unity do as simple work as possible and left the manipulation of frame inside the C#.

In the C# CustomProcessor.cs, the processingFrame is the callback function send to the CustomProcessingBlock instance, which will be invoked once you call the "block.ProcessingFrames(frames)". So what you want to do for your own filter should be implemented inside this processingFrame callback function, after you have done the manipulation for the frames, you can allocate new frames using the "FrameSource" instance then move the newly allocated frames to queue by calling the "source.FrameReady(frame)"

For more detailed information, please refer to the "cs-tutorial-3-processing" project inside the latest LibRS csharp wrapper.

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment] @Akthree Does the response help?