Running two GaussianSplatting model at the same time

[changruizhu96]

Hi Clarte,

Thank you for your excellent work on implementing Gaussian Splatting in VR. I'm currently trying to launch two models at the same time to perform some comparison tasks. For example, the first model launched is rendering the main scene while the second model is rendering in a small quad. My initial attempts failed because it was not possible to load both models at the same time. Can you give me some hints on how to solve this problem, e.g. which part of the code should be changed?

[Peamon]

Yes of course,

First you should create a map<int, GaussianSplattingRenderer> in PluginApi to create multiple gaussian splatting renderers. you should also create a map<int, std::vector<std::shared_ptr<POV>>> for povs.

Then you should change the interface :

• LoadModel should return an int (-1 if failed, the key of the map if ok)
• Alls other function (SetNbPov, SetPovParameters...) add, int model_number at the start of the function.

Then you should modify all the functions to take this new parameter in account.

In Draw function you should also draw each GaussianSplattingRenderer.

Finally you have to change the Csharp function in unity project in the same way.

For the quad rendering it should not be hard to change C# code in unity because plugin create an external texture that can be easily set as a material main texture. Be aware that rendering pov need a camera (for intrinsics and extrinsics parameters), event if you use the target texture on a quad.

It's a good idea, be free to make a pull request when you're done ;-) or ask.

[changruizhu96]

Thanks for your hints, I will have a try!

[changruizhu96]

Your hints above provide ideas on how to load and render both models at the same time. In addition to that, is it possible to create two Gaussian splatting game objects, following an existing C# script? (One rendering to the main camera and the other to a quad.) I guess this might not require any changes to the API. But Unity always errors out and then forces quit when I try this. Are there some rendering settings in the C# script that don't allow the rendering of two models?

[Peamon]

It’s because DLL is only loaded once so when you call DLL api from two c# it cause conflicts. You have to provide multi model support inside the DLL like I proposed it’s the only way.

[changruizhu96]

Thanks for letting me know!

[changruizhu96]

Hi, I devised a lazy way to do it, and it worked well when I tried it. Since DLL can be loaded only once, I duplicated the DLL and renamed it. Created another game object for the second Gaussian-splatting model.

[Peamon]

Well done 👍