[Question] Reproduction problem about Compactness-based Densification

[haodong2000]

Dear authors, thanks for your great work, but I met problems while training with the Compactness-based Densification proposed in section 4.2 of your paper, like the figure below:

I found that in your conf/base.yaml:

renderer:
  ...
  densify:
    enabled: true
    type: official
    ...
    use_legacy: true

, which means that in function def densify() at gs/gaussian_splatting.py:

    def densify(self, step, verbose=True):
        # check if densify is enabled and triggered, if true, do densify
        if not self.densify_cfg.enabled:
            return
        if step < self.densify_cfg.warm_up or step > self.densify_cfg.end:
            return

        if step_check(step, self.densify_cfg.period, True):
            if self.densify_cfg.use_legacy:
                self.densify_legacy(step, verbose)
                if "compatness" in self.densify_cfg.type:
                    self.densify_by_compatness(K=self.densify_cfg.get("K", 3))

, the function self.densify_by_compatness() will not be executed and only self.densify_legacy() will be ran.

And then I set renderer.densify.type="compatness", to run it, but the proformance is not quite good with prompt "a zoomed out photo of a 3D model of an adorable cottage with a thatched roof" with initial prompt "a cottage", at iter7000 (because after including the self.densify_by_compatness(), the training stopped at iter7000)

• with compatness

• without compatness

There might be several reasons, the first that I came up with that in defalult, the pruning operation is disabled in config. I will report the result after adding the pruning.

But the most important is authors' explaination, cause you are the most familiar with the code. Could you kindly provide some advice, since we are also doing Text-to-3D tasks, and the Compactness-based Densification really attracting us.

Thanks! 😄

[haodong2000]

Update: I've tried to enable pruning operation with below params:

renderer:
  ...
  prune:
    enabled: true
    warm_up: 3000
    end: 10000
    radii2d_thresh: 1000
    alpha_thresh: 1000
    radii3d_thresh: 0.0
    period: 500

, but after running self.prune_by_scale() & self.prune_by_alpha() at gs/gaussian_splatting.py, I met errors at trainer.py, line 299, in train_step: out = self.renderer(batch, self.cfg.use_bg, self.cfg.rgb_only). The logs is below.

terminal logs

``` Step 3000| 0 gaussians remaining ... | pruned by scale: 0| pruned by alpha: 163609| prune by radii3d 0 115710|2023-10-04|Iter: 3000/15000: 20%|████████████████████████████▊ | 3001/15000 [25:28<1:41:50, 1.96it/s, loss=679.5815] Error executing job with overrides: ['prompt.prompt=a zoomed out photo of a 3D model of an adorable cottage with a thatched roof.', 'init.prompt=a cottage', 'renderer.densify.type=compatness'] Traceback (most recent call last): File "/remote-home/share/data85/share/haodongli/code/gsgen/main.py", line 21, in main trainer.train_loop() File "/remote-home/share/data85/share/haodongli/code/gsgen/trainer.py", line 594, in train_loop loss += self.train_step() File "/remote-home/share/data85/share/haodongli/code/gsgen/trainer.py", line 299, in train_step out = self.renderer(batch, self.cfg.use_bg, self.cfg.rgb_only) File "/root/anaconda3/envs/gsgen/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1538, in _call_impl result = forward_call(*args, **kwargs) File "/remote-home/share/data85/share/haodongli/code/gsgen/gs/gaussian_splatting.py", line 1424, in forward self.render_one( File "/remote-home/share/data85/share/haodongli/code/gsgen/gs/gaussian_splatting.py", line 1273, in render_one T = torch.ones([H, W, 1], device=self.device, dtype=torch.float32) File "/root/anaconda3/envs/gsgen/lib/python3.9/site-packages/torch/utils/_device.py", line 62, in __torch_function__ return func(*args, **kwargs) RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. ```

Still figuring it out ... and thanks in advance for any help 😄

[heheyas]

Hi haodong,

Thanks for your interest. If you find the model contains an excessively large number of Gaussians, please set a smaller K by +renderer.densify.K=1 or a larger period on densification by renderer.densify.period=2000 which works fine in my case. Some of the configs have not been released yet since we want to do better (including guidance with DeepFloyd IF and VSD loss in ProlificDreamer).

The error that happened in the pruning is caused by the alpha_thresh: 1000 since alpha_thresh refers to the minimal opacity value thus all of the Gaussians are pruned. You can set the alpha_thresh=0.05 to enable a correct prune. Sorry for not making it clear in the readme. We are still working on GSGEN. If you have any more questions about the approach, feel free to email me.

[haodong2000]

Many thanks for your explaination! And sorry I did not set params like alpha_thresh and radii2d_thresh carefully :smile:

After adding the Compactness-based Densification, the details seems to be better!

• with compactness and prune

• with compactness

• without compactness