how ro reproduce garden result?

[zhouilu]

Nice work. I run garden scene, but still has artifact like origin 3d-gs.

python3 train.py -s ../data/garden -r 4 --data_device=cpu

So how to reproduce garden result, can you give some advices? thanks

[yileijin]

It's the results from both upscale diffusion model and the normal SD model. You need to download and turn on the upscale diffusion config to train your gaussians. More importantly, you have to finetune your SD model for more accurate context alignment, or the regenerated and amplified images tend to be very precarious in these artifacts. Finetuning strategies are in the supplementary material.

[yileijin]

Some additional explanations: these artifacts that appeared in the picture still "exist", but were modified. The optimization of the original 3D-GS is quite straightforward so there is no structural awareness of these points. In our method, the "white mass" artifacts are still there but through our regenerated new-view rendering, their colors are changed and become more natural. So if you want to have more "perfect" results, I recommend you to use Scaffold-GS to integrate our method. Its structural awareness is much better than the original 3D-GS.

[zhouilu]

thanks. i use sd_ckpt and default code(sd_config), here is my render result.

Does this meet expectations?

[yileijin]

This may meet the expectation. For the default mode on unfinetuned SD, you should compare the result to the original 3D-GS renderings. As we only add very small broken_strength (like 0.05 to 0.01) to these novel-view renderings, not many variations are reasonable. If you seek more natural ones but without an upscale diffusion model, you should finetune the diffusion model and use our V3-Experiment (broken strength 0.15-0.01) Setting.