search

facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data
https://pytorch3d.org/
Other
8.7k stars 1.3k forks source link

make PointTriangle3DistanceForward return barycentric coordinates #1865

Closed Kiord closed 1 week ago

Kiord commented 2 weeks ago

This issue has similarities with #193 and #1016

Hello Pytorch3D developers, thank you for this great library !

When looking the closest distance of query points on a mesh, it can be useful to get the nearest point and/or the barycentric coordinates on the support triangle. For example, in Trimesh, there is trimesh.proximity.closest_point:

closest_points, distance, triangle_ids = trimesh.proximity.closest_point(mesh, points)

Its equivalent in Pytorch3D seems to be loss.point_mesh_distance.point_face_distance but it only provides the distance and closest triangle id, not the nearest point nor its barycentric coordinates.

My request is that PointFaceDistanceForward would also retrieve the nearest point and/or the barycentric coordinates. This would be nicer/faster than recomputing the nearest points from the triangles indices, right ?

This implies modifying at least PointTriangle3DistanceForward and PointLine3DistanceForward from utilitary functions. So this should maybe be implemented in new functions to not break anything...

This should be doable for the forward functions, but I have no idea regarding the gradient computation. Do you think this is feasible ?

bottler commented 1 week ago

I wonder whether it is possible to trick the mesh rasterizer into doing what you want without much code change. That would be quite a lot easier.

Kiord commented 1 week ago

Actually, I do not work with the mesh rasterizer, I just needed the closest point on a mesh differentiably. I ended up using _C.point_face_dist_forward and recomputing the barycentric coordinates and closest points as mentionned in #1016. However, _C.point_face_dist_forward already internally computes the barycentric coordinates and closest points. I understood that the bottleneck in the closest point query is finding the corresponding triangles, so recomputing them does not matter much. If you think that this feature request is not relevant enough you may close this issue.

bottler commented 1 week ago

I think I will close as you have a workaround.