yzslab / gaussian-splatting-lightning

A 3D Gaussian Splatting framework with various derived algorithms and an interactive web viewer
Other
457 stars 38 forks source link

modfied blender data parser #54

Closed hardikdava closed 2 weeks ago

hardikdava commented 1 month ago

@yzslab This PR enables following features.

yzslab commented 1 month ago

Does the NGP dataparser work as you expected? One reason that I did not introduce it in the README because I have not really tested it. Another problem is that most of the NGP datasets contain camera distortions. Theoretically this can be solved with the undistortion functions provided by OpenCV. But comparing with the colmap undistorter, OpenCV does not perform well on the experiment metrics. So I hid it in case someone use it without being aware of the distortion problem.

hardikdava commented 1 month ago

@yzslab yes, you are right. But nerfstudio uses cv2.undistort function to undistort images and it is working very well.

yzslab commented 1 month ago

@yzslab yes, you are right. But nerfstudio uses cv2.undistort function to undistort images and it is working very well.

When did you test it, and how well it worked?

It has been a confirmed issue since January: https://github.com/nerfstudio-project/nerfstudio/issues/2849, and was only claimed to have been fixed two weeks ago: https://github.com/nerfstudio-project/nerfstudio/pull/3382. But not sure whether it was fixed really because they did not provide any metrics. Even the team members of NeRFStudio can not clearly tell what makes such a difference.

yzslab commented 1 month ago

If you really need it, and know how to deal with the distortion, it still can be enabled with the full class path, e.g., --data.parser internal.dataparsers.ngp_dataparser.NGP, without being added to __init__.py.

Besides, there is a better way to dealing with the NGP dataset. You can convert it back to colmap sparse model like utils/meganerf2colmap.py. Not only can you utilize colmap's undistorter but also a better point cloud that some of the NGP datasets do not contain.

hardikdava commented 4 weeks ago

If you really need it, and know how to deal with the distortion, it still can be enabled with the full class path, e.g., --data.parser internal.dataparsers.ngp_dataparser.NGP, without being added to init.py.

Thanks.

Besides, there is a better way to dealing with the NGP dataset. You can convert it back to colmap sparse model like utils/meganerf2colmap.py. Not only can you utilize colmap's undistorter but also a better point cloud that some of the NGP datasets do not contain.

I was not aware of such method. I guess then we can add this lines into docs so users can use it well only for undistorted images. I will revert my changes for NGP dataparser.

hardikdava commented 4 weeks ago

@yzslab Please review this PR.

yzslab commented 2 weeks ago

Thanks!