search

zinsmatt / SpaceCarving

36 stars 4 forks source link

Some questions about the algorithm #7

Open ChauChorHim opened 2 years ago

ChauChorHim commented 2 years ago

I am beginner in computer graphics and confused about some details of the algorithm. Here is my overall understanding:

  1. create a cube, which should embrace the object we want to reconstruct.
  2. using the projection matrix in each view to project the object 3D points into the image plane and get a long array 'fill'. This 'fill' indicate if each pixel in the image plane is in the silhouette.
  3. Stack all the 'fill', we can vote for each voxel in the original cube to tell if it belond to the 3D structure we want to reconstruct.

Still, I have some specific question.

  1. Why do you subtract the z-axis of pts with 0.62?
  2. As far as I know, projection matrix is used to get the image points from the world points. And in your code, the projection matrix is multiplied with the 3D points in the cube cooridnates. So the projection matrix is not like the normal one or the world coordinate is the same as the cube coordinates?
  3. Is 'uvs' the projected image points at each view? Why some of the pixels will outside the image boundary after using the projection matrix?

Also, could you suggest some reference to make these cloud points into mesh? Thank you!

zinsmatt commented 2 years ago

Hello,

Your understanding of the algorithm seems globally correct. Just to clarify, the points that are projected in the images are not the points of the object, but the points of a "virtual" grid that is created and adjusted to be roughly located at the place of the object. Initially, we do not have any 3D object. The size of the 3D grid limits the area of reconstruction and its number of subdivisions (its resolution) influences the quality and the level of details of the reconstruction. In the code, pts correspond to the centers of the grid voxels.

  1. Subtracting 0.62 was just done to adjust the grid to the object that I reconstructed (so that the grid contains the entire object)
  2. Yes, in that simple case the grid is directly defined in world coordinates.
  3. uvs are the points projected in one image, but this operation is then repeated on each image (see the loop starting line 65). There is no guarantee that these "virtual grid points" always projects inside the image. It is possible that some of them are just not visible in some image.

ParaView is a great tool for that. At the end of the code the occupancy grid is saved as .vtr file. You can open it in ParaView and add the filters "Iso Volume" (you can set the threshold value to separate "empty" and "filled" voxels wrt. the occupancy value in the grid; in the example a threshold around 28 produces nice results) and "Extract Surface". You can then save the mesh in many formats PLY, OBJ, STL, ...

Matthieu