The training results are a little vague.

[diamond0910]

Thank you for your great work. I finish the whole training process on paper-windmill dataset using 7 hours. But from the results, the rendered image is not clear enough.

This is some results from the training view.

And this is from the val view.

But I see the pictures in your paper and the video on the homepage are of good quality. Do you have any problems with my training? Can you give me some advice?

[coltonstearns]

Hmm let me look into this. On the DyCheck scenes, the training and validation views do look notably worse than many of our teaser videos on the Nvidia dataset and real world videos because (1) the DyCheck provided depth is lower-res and noisier, and (2) the validation novel view is at times quite different from the training view, so we use larger tracking and rigidity losses to further regularize the 3D scene, but this will lead to worse training-view reconstruction.

However, these results do look a little worse than I'd expect. Are the quantitative PSNR and LPIPs metrics on-par with what was reported in the paper?

[diamond0910]

Sorry, the log information is not saved. I ran the following command and got the results.

python train.py --data_dir ./data/nvidia/Balloon1 --config configs.dgmarbles_nvidia --load_dir ./checkpoints/balloon1 --outdir ./out --only_render True

wandb:       Dycheck-MeanDist 8.87307
wandb:    Dycheck-MeanSuccess 0.94444
wandb:          FINAL_VAL_fps 187.50169
wandb:        FINAL_VAL_lpips 0.48587
wandb: FINAL_VAL_nomask_lpips 0.50108
wandb:  FINAL_VAL_nomask_psnr 17.03427
wandb:  FINAL_VAL_nomask_ssim 0.35625
wandb:         FINAL_VAL_psnr 17.02651
wandb:         FINAL_VAL_ssim 0.37906

My training command is python train.py --data_dir ./data/paper-windmill --model_config configs.dgmarbles_dycheck_wpose --outdir ./out. After 6 hours of training, I got the step-000327967.ckpt file.

Looking forward to your reply.

[coltonstearns]

Hi, I'm very sorry about forgetting to respond to this. Did you ever resolve this? Looking at the metrics, those numbers are indeed notably worse than what I would expect. On my side, (1) I will try to reproduce the segmentation fault with the CUDA 11.7 setup, and see there is an issue with the install, and (2) I am about to release an updated version of the code that should bring better training and even better results on the DyCheck data.