hustvl / 4DGaussians

[CVPR 2024] 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering
https://guanjunwu.github.io/4dgs/
Apache License 2.0
2.25k stars 187 forks source link

About the translation of camera in HyperNeRF dataset and coords convention #95

Open MobiusLqm opened 9 months ago

MobiusLqm commented 9 months ago

Hi, Thanks for your wonderful work.

  1. in your codes line 164-165 of scene/hyper_loader.py , you useT = - camera.position @ R instead of T = camera.position,Could your please tell What's the reason behind it?
  2. Addtionaly, as far as i know , 3d gaussian splatting uses opencv coords convention, but hypernerf dataset uses openGL convention. Why don't you convert it to opencv coords? Looking forward to your reply! Regards.
guanjunwu commented 9 months ago

Thanks for your question!

  1. I've tried many times to align camera poses among these datasets. Maybe the camera pose of hypernerf is c2w instead of w2c. So I have to change it.
  2. In fact ,the full loader code is borrowed from TiNeuVox. And the colmap.sh also generate dense point clouds from this dataset, in order to make it useful and simple, so I didn't convert it. When running the hypernerf dataset, you can found a image will be saved as 'output.png', that's my debug figure that point out the relationship between camera poses and point clouds.
MobiusLqm commented 8 months ago

Thank your for you quick reply! https://github.com/google/nerfies#datasets

  1. after checking the repo of nerfies and hypernerf, I found that hypernerf follows nerfies dataset format. The orientation is a w2c matrix, and the position is in world coords. This makes me more confused , I still dont understand why this line of code works.
  2. So sorry, my bad, Hypernerf uses opencv coords convention.