Question about densification with split and clone

[leizhenyu-lzy]

In the source code shown below I can see a bias is add to the new_xyz when doing split, but it seems that when doing clone new gaussian is just using the same properties, it doesn't look like the image shown in the paper(shown below), and will these 2 gaussians do the same thing(transform in the same way) in the training process and the result will not be the same shown in the picture below?

Thank you for your help and reply.

    def densify_and_split(self, grads, grad_threshold, scene_extent, N=2):
        n_init_points = self.get_xyz.shape[0]
        # Extract points that satisfy the gradient condition
        padded_grad = torch.zeros((n_init_points), device="cuda")
        padded_grad[:grads.shape[0]] = grads.squeeze()
        selected_pts_mask = torch.where(padded_grad >= grad_threshold, True, False)
        selected_pts_mask = torch.logical_and(selected_pts_mask,
                                              torch.max(self.get_scaling, dim=1).values > self.percent_dense*scene_extent)

        stds = self.get_scaling[selected_pts_mask].repeat(N,1)
        means =torch.zeros((stds.size(0), 3),device="cuda")
        samples = torch.normal(mean=means, std=stds) 
        rots = build_rotation(self._rotation[selected_pts_mask]).repeat(N,1,1)
        new_xyz = torch.bmm(rots, samples.unsqueeze(-1)).squeeze(-1) + self.get_xyz[selected_pts_mask].repeat(N, 1)  # add bias
        new_scaling = self.scaling_inverse_activation(self.get_scaling[selected_pts_mask].repeat(N,1) / (0.8*N))
        new_rotation = self._rotation[selected_pts_mask].repeat(N,1)
        new_features_dc = self._features_dc[selected_pts_mask].repeat(N,1,1)
        new_features_rest = self._features_rest[selected_pts_mask].repeat(N,1,1)
        new_opacity = self._opacity[selected_pts_mask].repeat(N,1)

        self.densification_postfix(new_xyz, new_features_dc, new_features_rest, new_opacity, new_scaling, new_rotation)

        prune_filter = torch.cat((selected_pts_mask, torch.zeros(N * selected_pts_mask.sum(), device="cuda", dtype=bool)))
        self.prune_points(prune_filter)

    def densify_and_clone(self, grads, grad_threshold, scene_extent):
        # Extract points that satisfy the gradient condition
        selected_pts_mask = torch.where(torch.norm(grads, dim=-1) >= grad_threshold, True, False)
        selected_pts_mask = torch.logical_and(selected_pts_mask,
                                              torch.max(self.get_scaling, dim=1).values <= self.percent_dense*scene_extent)

        new_xyz = self._xyz[selected_pts_mask]
        new_features_dc = self._features_dc[selected_pts_mask]
        new_features_rest = self._features_rest[selected_pts_mask]
        new_opacities = self._opacity[selected_pts_mask]
        new_scaling = self._scaling[selected_pts_mask]
        new_rotation = self._rotation[selected_pts_mask]

        self.densification_postfix(new_xyz, new_features_dc, new_features_rest, new_opacities, new_scaling, new_rotation)

[AsherJingkongChen]

Either those small splats or remaining splats are treated in the same way in the next iteration or later. I think those small splats can have 0.5 std bias, which would make them different.

[djx99]

Yes, it seems that when clone, the original gauss is copied exactly new_xyz = self._xyz[selected_pts_mask] and an offset is not added. Can anyone explain this issue?

Either those small splats or remaining splats are treated in the same way in the next iteration or later. I think those small splats can have 0.5 std bias, which would make them different.

Why do they have 0.5 std bias, what does 0.5 std bias mean? Thank you very much.

[AsherJingkongChen]

Why do they have 0.5 std bias, what does 0.5 std bias mean? Thank you very much.

@djx99 I forgot that the original implementation used different deviations on every point. I simplified that by using std = 1.0. This context should be irrelevant to the original questions.

I think cloning can have lower deviation than splitting, but there should be no big difference since scales are small.

You can try it by copying parts of splitting and multiply a real number less than 1.0.

[djx99]

Thank you for your reply, I have just started learning 3DGS and had the above question when reading about this function. If I have time, I will try it; I don't think it will affect the results too much. Thanks again.