search

marian42 / mesh_to_sdf

Calculate signed distance fields for arbitrary meshes
https://pypi.org/project/mesh-to-sdf/
MIT License
991 stars 107 forks source link

Cannot generate correct SDF results for car objects in ShapeNet-v2 #12

Closed YuDeng closed 4 years ago

YuDeng commented 4 years ago

Hi, I've test the sample_sdf_near_surface scripts using car objects in ShapeNet-v2 dataset. However, I found that the sign of some points can be wrong for almost every car object in the dataset. For example, 1

I plot the SDF for the above car model in Matlab to see if it is correct: in the first figure, I plot all the points with negative SDF value; and in the second figure, I plot all the points with their absolute value less than 0.01: 2 3

Clearly, if I only plot those points that are close enough to the surface, everything seems fine, which suggests that the script does give correct results for point-to-surface distances. However, it may give the wrong sign for some points as shown in the second figure.

I use the "scan" method and "normal" sign_method. And this happens for almost every car object. If I switch the sign_method to "depth", it won't give negative SDF value for points outside the object but will return much fewer points with negative SDF value, which indicates that some inner points receive positive SDF values incorrectly.

I've checked your implementation and noticed that the wrong sign may be due to some incorrect vertex normals. I also found that the car objects in ShapeNet-v2 do have some double faces and inward face normals. I wonder if this influences the rendering process to obtain correct SDF? Have you ever met the same problem for car objects in ShapeNet? Or do you know how to fix this problem?

Looking forward to your reply.

marian42 commented 4 years ago

Sorry for the trouble. I'm aware of the problems you mention. For the scan method, the algorithm generates a surface point cloud. Sometimes, few points in the surface point cloud have bad normals. Each of those regions, usually caused by a small triangle with bady oriented normals, will cause a "cone" of incorrect signs.

Both algorithms should handle double faces and flipped normals (when using the "scan" instead of "sample"). The artifacts in your screenshot come from tiny spikes in the mesh. I'm not sure how to handle that unfortunately (I have some ideas, remeshing could help...)

If I switch the sign_method to "depth", it won't give negative SDF value for points outside the object but will return much fewer points with negative SDF value, which indicates that some inner points receive positive SDF values incorrectly.

This could be due to holes in the model. The depth method assigns positive signs to all points that are seen by at least one of the scan cameras. To debug this issue, it might be helpful to create a similar plot for the depth method. In general, the depth method avoids the "spike" artifacts that you encounter with the "normal" method, but creates slightly noisier surfaces.

When working with Shapenet, I suggest that you discard the shapes with bad SDFs. In my case, I could keep slightly more than half of the shapes. This repository has code to check if a voxel volume has plausible (continuous) SDFs. You could implement the same for point clouds.

aluo-x commented 3 years ago

I've been looking into this library recently, and at the Shapenet v2 dataset. Unfortunately even simple classes (chair, table) have a significant amount of meshes that are poorly behaved, and the "normal" method for determining sign is mostly useless on these poorly behaved meshes.

The "depth" method that the author came up with gives good results in most cases. Once the deadlines are over, I may give a shot at the SDF generation problem.