dcharatan / pixelsplat

[CVPR 2024 Oral, Best Paper Runner-Up] Code for "pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction" by David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann
http://davidcharatan.com/pixelsplat/
MIT License
830 stars 56 forks source link

Hello, the point cloud visualization image in your paper is very beautiful. How was it drawn? #71

Closed chenkangjie1123 closed 4 months ago

chenkangjie1123 commented 4 months ago
截屏2024-05-08 00 24 04

Hello, the point cloud visualization image in your paper is very beautiful. How was it drawn? I tried to use the generate_point_cloud_figure.py file in the code to draw it, but the results were quite different.

The following is drawn by generate_point_cloud_figure.py

000000_9e2a8cc5f32dd46b_angle_076

The following is the corresponding scenario

000075_gt

chenkangjie1123 commented 4 months ago

000000_9e2a8cc5f32dd46b_angle_100

It is difficult to find a great angle

dcharatan commented 4 months ago

Since each scene has a different scale, the figure generation requires a bit of manual tweaking when you use it for a new scene. You'll have to adjust the far clipping plane to only render the correct region of space, then pick a good-looking angle for the point cloud. You can adjust these things here:

SCENES = (
    # scene, context 1, context 2, far plane
    # ("fc60dbb610046c56", 28, 115, 10.0),d7c9abc0b221c799
    # ("1eca36ec55b88fe4", 0, 120, 3.0, [110]), # teaser fig.
    ("2c52d9d606a3ece2", 87, 112, 35.0, [105]),
    ("71a1121f817eb913", 139, 164, 10.0, [65]),
    ("d70fc3bef87bffc1", 67, 92, 10.0, [60]),
    ("f0feab036acd7195", 44, 69, 25.0, [125]),
    ("a93d9d0fd69071aa", 57, 82, 15.0, [60]),
)

In general, I found that indoor scenes with large furniture objects and walls on 2 sides (rather than 3) look best and highlight the 3D-ness of the point cloud the most.

chenkangjie1123 commented 4 months ago

Thank you very much for your reply. I have successfully experimented this morning. This angle parameter is indeed difficult to adjust, and the quality of different scenes varies greatly.

dcharatan commented 4 months ago

The noise in the 3D point clouds is indeed a limitation of pixelSplat. If you want cleaner point clouds, it might be worth checking out MVSplat, which builds on top of pixelSplat and produces much cleaner point clouds!