search

turandai / gaussian_surfels

Implementation of the SIGGRAPH 2024 conference paper "High-quality Surface Reconstruction using Gaussian Surfels".
501 stars 24 forks source link

BlendedMVS evaluation script #7

Closed hanyucc closed 4 months ago

hanyucc commented 4 months ago

Could you share the evaluation script you use to produce the BlendedMVS results? I assume it should be pretty similar to the DTU evaluation script, but since the ground truth data formats are pretty different, I'm wondering if you could share the actual script that you used to obtain the BlendedMVS results. I noticed that there is a function eval_bmvs commented out in the code but don't see it defined anywhere. Thanks!

turandai commented 4 months ago

Hi, the evaluation for BlendedMVS is straightforward: 1) we sample points from mesh as the DTU evaluation code does, 2) scaling up the two point cloud for x1000, as the original BlendedMVS data is normalized approximately in a bounding sphere of size 1, the factor of 1000 roughly match the scales of BlendedMVS scenes with DTU, 3) then calculate the Chamfer distance between sampled points and the ground truth point cloud as the DTU evaluation code does.

hanyucc commented 4 months ago

Great! Thanks for the info. I was able to get the DTU evaluation script to work, but the Chamfer distances I got were significantly lower than those reported in the paper. Scaling the point clouds x1000 should be what I'm missing.

hanyucc commented 4 months ago

Actually, just to double check, do you use the same parameters as DTU in the evaluation script? In particular, I found that after scaling the point cloud, using the default downsample_density of 0.2 results in a huge point cloud with over 4 million points... Is this expected or am I doing something wrong?

turandai commented 4 months ago

Hi, we compute the Chamfer distance after sampling points from mesh without further downsampling for BlendedMVS.

hanyucc commented 4 months ago

I believe the sampling density of the initial set of points also depends on the downsample_density parameter (link). Do you use the default value for that? Also, for other parameters like max_dist, which is used to determine and ignore outliers in the point cloud, do you also use the default values? I'm mostly asking because the scaled up meshes are a bit larger in scale (maybe 4x?) than the DTU meshes, so I'm wondering if you scale up those parameters accordingly as well. Thanks!

turandai commented 4 months ago

I am sorry for the confusion on the scaling, I have updated the evaluation script here.

hanyucc commented 4 months ago

Thanks, that's really helpful