search

NVlabs / CF-3DGS

Other
325 stars 35 forks source link

Fail to reproduce (Co3D dataset). Ques: Image resolution + Pretrained model #7

Open WalterC9 opened 3 months ago

WalterC9 commented 3 months ago

Hi, thank you very much for sharing your code. I have several questions regarding your work:

  1. Could you specify the image resolution used during training for each scene in the Co3D dataset?

  2. Would it be possible for you to share your pretrained Gaussians from the Co3D dataset? This would be very helpful for comparing qualitative results between different methods.

  3. I encountered an out-of-memory issue while trying to reproduce the results on the Co3D dataset (though it works fine for the Tanks dataset). Your paper mentions that a single RTX 3090 is sufficient to train the model, so I expected the code to work on my RTX 4090. Did you face this issue during your training? If so, how did you resolve it?

Your help is greatly appreciated!

OasisYang commented 3 months ago
  1. In most cases, we used the original resolution for scenes in the CO3D dataset.
  2. Sure, I'm trying to find the rendered RGB images used in our paper and will upload them as soon as possible. To reproduce the results shown in our paper, please use the ZoeDepth as the depth estimator
  3. I didn't face this problem for experiments on CO3D and Tanks&Temples, but I got this issue when running on some internet videos. This may caused by the inaccurate pose estimation as we didn't optimize the pose along with the optimization of the global gaussian model. Our follow-up work resolves this issue by a joint optimization on camera poses and 3D gaussian. Feel free to play with it.
WalterC9 commented 2 weeks ago

Hi OasisYang,

Thank you for releasing the rendered images and depth maps for visual comparison — I greatly appreciate it!

However, I am having difficulty in (1) drawing poses visualization and (2) evaluating the rendered depth; as only the depth images are released instead of actual depth value.

I was wondering if it would be possible to also release the optimized Gaussians and poses. This would be very helpful to compare different model both quantitatively and quantitatively.

Kevin-2017 commented 4 hours ago

I have the same issue with reproducing the Co3d results. Has anyone else been able to reproduce the results? What is the hyperparameter used for co3d? Below is the command that I used. 

python run_cf3dgs.py \
      -s "path" \
      -m "output/${scene}" \
      --mode train \
      --data_type co3d \
      --eval