Pass covariance to the render?

[fishfishson]

Hi author,

Is there anyway to pass cov directly to the render rather than the scale and rotation?

[oliver-batchelor]

Hi @fishfishson,

To do this you would want to make a new projection function which directly takes the 3D covariance. Since the projection function and rasterization functions are separate (rasterization takes 2D gaussians as input). you can make your own top level render_gaussians and projection function which accepts directly the 3D covariance, because this work uses the taichi autograd it will be a very simple alternative version of the project_perspective_kernel: https://github.com/uc-vision/taichi-splatting/blob/2c1f129b130c70a03cf57f0d589ce59f38af7e3b/taichi_splatting/perspective/projection.py#L30

I think this is the simplest way to achieve this. If you get stuck I am happy to help.

[fishfishson]

Sorry for late reply. Thx for your solution! It works.