search

Anttwo / SuGaR

[CVPR 2024] Official PyTorch implementation of SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering
https://anttwo.github.io/sugar/
Other
2.12k stars 152 forks source link

unexpected results #42

Open PGogo opened 8 months ago

PGogo commented 8 months ago

Hi, i'm trying to reproduce your excellent mesh reconstruction. But got an unexpected result instead. I used the data with colmap provided by the original 3D Gaussian Splatting code. And obtain the truck as (open in Blender) image

(open in meshlab) image

My commands: python gaussian_splatting/train.py -s ./truck/ --iterations 7000 -m output/truck python train.py -s truck/ -c output/truck/ -r density

I had try "-r sdf" and the result was about the same. The defect is also exists in other reconstructions like playroom.

Could you give me some suggestions? Thanks!

XOURTNEY commented 8 months ago

I think like NeRF with maching cubes,they cannot reconstruct transparent object, 3DGS with SuGaR cannot reconstruct transparent object, neither. Both NeRF and 3DGS reconstruct transparent things in a hallucinated way.

PGogo commented 8 months ago

I think like NeRF with maching cubes,they cannot reconstruct transparent object, 3DGS with SuGaR cannot reconstruct transparent object, neither. Both NeRF and 3DGS reconstruct transparent things in a hallucinated way.

Actually, the defects I mentioned above refer to the pits on the surface of the car's hood. The surface of the mesh is not smooth enough as the demo of sugar. It will be great if @Anttwo could share the original data (images and colmap) of the demos like "dukemon" on a google drive or somewhere.

XOURTNEY commented 8 months ago

Hahaha, sorry about my misunderstood. Looking forward to Anttwo's reply either!

sky3 commented 8 months ago

Still early to show this as many things are still work in progress, quite advanced. Sill in progress but 3DGS definitely handle transparency, .... standard 3D reconstruction does not. I have this sample I can share to you with complete 3DGS implementation. https://app.inspacer.checkandvisit.com/projects/8897bb1d-05e4-422b-a17d-fb987d97eaff Select ultra quality. Best,

PGogo commented 8 months ago

Still early to show this as many things are still work in progress, quite advanced. Sill in progress but 3DGS definitely handle transparency, .... standard 3D reconstruction does not. I have this sample I can share to you with complete 3DGS implementaiton https://app.inspacer.checkandvisit.com/projects/8897bb1d-05e4-422b-a17d-fb987d97eaff Select ultra quality. Best,

So kind you are! It seems that the url is unavailable? Maybe it is caused by my late response and I am sorry for that. Can you upload it again?

XOURTNEY commented 8 months ago

Still early to show this as many things are still work in progress, quite advanced. Sill in progress but 3DGS definitely handle transparency, .... standard 3D reconstruction does not. I have this sample I can share to you with complete 3DGS implementaiton https://app.inspacer.checkandvisit.com/projects/8897bb1d-05e4-422b-a17d-fb987d97eaff Select ultra quality. Best,

I think that while we can see the transparency through the 3DGS viewer, that doesn't mean we've reconstruct its mesh. If you look through the .obj we construct, it's missing sometimes.

sky3 commented 8 months ago

I have just updated the markdown on my previous comment, the url was fine , the markdown link was not... Sorry. That's a real thing, 3DGS allow to minimize efficiently appearance to deal with both transparency and reflectivity. Car windows have to model both... the real thing is which material representation do you aim to retrieve and then how do you make sure to sample efficiently your windows to represent a double sided geometry or not. Floating 3d gaussian are incredibly good to deal with such problems.