Clarification on Pre-trained PLY Requirement for DTU and MipNeRF 360 Training Scripts

[hwanhuh]

First of all, thank you for your great work!

I've been exploring your training scripts for DTU and MipNeRF 360 data. Unless I misunderstand, it seems that both scripts require a pre-trained PLY file from either vanilla 3D GS or 2D GS.

I have a couple of questions regarding this:

1. Is a pre-trained PLY file essential for training on DTU and MipNeRF 360 data?
2. What is the expected performance of trim2dgs/3d gs from scratch compared to using a pre-trained PLY file with 7K iterations?

I appreciate your guidance on this matter.

[hwanhuh]

Additionally, I have a question about the depth_grad_weight used for normal consistency loss. The weight seems to be designed to penalize only the regions with 'large changes' in surface depth. However, I couldn't find the exact reasoning for this weight in your arXiv paper. Could you please explain the rationale behind this design choice?

[Abyssaledge]

Thank you so much for your attention. The pretrained 3DGS/2DGS is not essentially necessary. This is equivalent to training a model from scratch with enabling our strategy at a later stage (for example, after 20000 iterations). The reason why we utilize the pretrained files is to accelerate the experiments, so that we could train models for only the last 7k iterations for each experiment, which saves a lot of time for experiments. In this sense, there is no performance difference between training from scratch or pretrained files.

[Abyssaledge]

For your next question, with the depth_grad_weight, we apply weaker penalty to those sharp regions instead of "penalize only the regions". Perhaps you misunderstood and got the opposite meaning :).

Specifically, we use the additional depth_grad_weight to avoid blurring the normal value in sharp object boundaries. The normal is estimated from a small sliding window. Thus, the normal tends to be smoothed if the sliding window is placed at the sharp boundaries. We will update the manuscript to add this detail. Thank you so much for pointing it out!

[hwanhuh]

Thanks for your reply. Then, is the training time of the 'tune.py' from scratch similar to the vanilla 3D GS/2D GS?

• oh, I miss "-1 x F.max_pool2d" Thanks!

[Abyssaledge]

Of course, before enabling our strategy, the training is the same as 3D/2DGS.

[hwanhuh]

What is the difference in teams of quality between scale and mix in densification? Could you share the results of your experiment?

[Abyssaledge]

There is no significant difference. The mix mode produces more Gaussians, so the rendering quality will be slightly higher for scenes with more frequent details, such as MipNerf360. For compactness, we still use the scale mode for MipNerf360 in our paper. We will update the performance comparison here when we are free. @hwanhuh

[hwanhuh]

thank you foe your kindness reply!

[YuxueYang1204]

Hi, @hwanhuh ! To differentiate scale and mix mode, we attach performance of reconstructed meshes and the storage consumption of .ply files on DTU here.

Mesh

	24	37	40	55	63	65	69	83	97	105	106	110	114	118	122	Mean
Trim2DGS (scale)	0.53	0.81	0.43	0.53	1.12	0.78	0.76	1.19	1.15	0.61	0.66	0.99	0.39	0.62	0.49	0.74
Trim2DGS (mix)	0.45	0.72	0.33	0.40	0.97	0.72	0.73	1.21	1.14	0.61	0.67	1.01	0.41	0.60	0.44	0.69

Storage (MB)

	24	37	40	55	63	65	69	83	97	105	106	110	114	118	122	Mean
Trim2DGS (scale)	18	17	30	10	2	10	11	39	24	23	20	26	21	18	18	19
Trim2DGS (mix)	42	39	49	30	22	21	24	40	47	34	33	34	31	32	34	34