PLY Output Format & Isotropic Gaussians

[ironjr]

Thanks for sharing the great project! I have run the code for iPhone demo and it worked smooth and nice.

However, I have two questions regarding the output format:

1. The current output of the project is a numpy zipped file of the combined 3DGS parameters + trajectory params.npz, not directly compatible to .ply files used in many Gaussian splatting works.

For this, I just made a converting code below, which produces gsplat.ply file compatible for the most of gsplat applications. Can you consider adding this kind of code in your project?

import numpy as np
from plyfile import PlyElement, PlyData # Requires plyfile==0.8.1

def construct_list_of_attributes(f_dc, scale, rotation):
    l = ['x', 'y', 'z', 'nx', 'ny', 'nz']
    for i in range(f_dc.shape[1]):
        l.append('f_dc_{}'.format(i))
    l.append('opacity')
    for i in range(scale.shape[1]):
        l.append('scale_{}'.format(i))
    for i in range(rotation.shape[1]):
        l.append('rot_{}'.format(i))
    return l

def convert(src, dest):
    params = np.load(src)

    xyz = params['means3D']
    normals = np.zeros_like(xyz)
    f_dc = params['rgb_colors']
    # f_rest = np.zeros_like(f_dc)
    opacities = params['logit_opacities']
    scale = params['log_scales'].repeat(3, axis=-1)
    rotation = params['unnorm_rotations']

    dtype_full = [(attribute, 'f4') for attribute in construct_list_of_attributes(f_dc, scale, rotation)]

    elements = np.empty(xyz.shape[0], dtype=dtype_full)
    attributes = np.concatenate((xyz, normals, f_dc, opacities, scale, rotation), axis=1)
    elements[:] = list(map(tuple, attributes))
    el = PlyElement.describe(elements, 'vertex')
    PlyData([el]).write(dest)

if __name__ == '__main__':
    src = 'params.npz'
    dest = 'gsplat.ply'
    convert(src, dest)

2. When I opened the params.npz, it has isotropic scaling of each Gaussians. Can I get anisotropic Gaussians instead? Does it result in worse results?

[Nik-V9]

Hi, Thanks for trying out our code and for this amazing enhancement!

1. Please open a pull request and feel free to add an example use of the .ply format on the README. Happy to incorporate it ourselves if you think that's better.
2. Yes, we currently use isotropic scaling. It should be easy for us to support anisotropic Gaussians in Pytorch (we need to add a few configs across the code, adding back our old code). I'll add this to our V2 list. We don't see any performance discrepancies for isotropic vs anisotropic. The nice thing about isotropic is that it has fewer parameters, so if we implement the CUDA operations accordingly, it should be faster and have lower memory. Hopefully, we plan to have custom CUDA implementations soon.

[ironjr]

Thank you for your thoughtful consideration. I have opened a pull request. https://github.com/spla-tam/SplaTAM/pull/29

By drag and dropping .ply file generated in the same directory as params.npz into any GS viewers, for example, https://playcanvas.com/super-splat you will visualize / edit the Gaussians generated by your algorithm.

[Buffyqsf]

Hi, Thanks for trying out our code and for this amazing enhancement!

1. Please open a pull request and feel free to add an example use of the .ply format on the README. Happy to incorporate it ourselves if you think that's better.
2. Yes, we currently use isotropic scaling. It should be easy for us to support anisotropic Gaussians in Pytorch (we need to add a few configs across the code, adding back our old code). I'll add this to our V2 list. We don't see any performance discrepancies for isotropic vs anisotropic. The nice thing about isotropic is that it has fewer parameters, so if we implement the CUDA operations accordingly, it should be faster and have lower memory. Hopefully, we plan to have custom CUDA implementations soon.

Hey, thank you for your work. I am reading the code these days. I want to know how do you imply isotropic in code. Is that because you use torch.tile() to copy scale? So that all three directions' scales will have the same grad?

[Nik-V9]

Hi, Yes. Since the Gaussians are isotropic (spherical), we only model one parameter for scale across x, y & z instead of using 3 parameters.

As you mentioned, currently, we pass the Gaussians using a tile operation to be compatible with the anisotropic CUDA rendering from the official 3DGS code, as shown here: https://github.com/spla-tam/SplaTAM/blob/bbaf5cc5754bf1034b33902007872c694e412a31/utils/slam_helpers.py#L112

Furthermore, the isotropic Gaussians are defined here: https://github.com/spla-tam/SplaTAM/blob/bbaf5cc5754bf1034b33902007872c694e412a31/scripts/splatam.py#L141 The dimension of the log_scales is [num_gaussians, 1].

[Buffyqsf]

@Nik-V9 I got it. Thanks!