the out.ply from mesh_reconstruction is not matching original object's point cloud

[qyp-robot]

when I go to test, I launch a coke can object of single view point cloud,like this: But, at the end, the out.ply from mesh_reconstruction executable file，which cannot match the original object, it likes a Cube shape, not a coke can shape: what troubles this process happens？very eager for your help! thank you ! @jvarley

[qyp-robot]

@DavidWatkins

[qyp-robot]

the “rosrun pc_object_completion_cnn mesh_completion_server.py depth” window see as follow：

[DavidWatkins]

Did you run it through the neural network with trained weights? That looks like the result of using an untrained version of the model.

[qyp-robot]

I only use the command: "rosrun pc_object_completion_cnn mesh_completion_server.py depth" to run the neural network, my trained models looks like this: how can I to change the trained weights,now,,it looks like default parameters

[qyp-robot]

I search the trained weights in " mesh_completion_server.py" code,this looks like: but I run the code with default form "rosrun pc_object_completion_cnn mesh_completion_server.py depth",I donot know where to change the trained weights. thank you !best wishes!

[qyp-robot]

@DavidWatkins

[qyp-robot]

I print the trained weights path, it looks this: this path matchs my trained weights "best_weights.h5"; I believe this path is true,what can I to detect the bug? thank you ! @DavidWatkins

[qyp-robot]

I test other shapes object ,the output are the same mesh, I guess the neural network not work normally @DavidWatkins . very appreciate you!

[qyp-robot]

and the cnn model is shape_completion_server.trained_models.depth_y17_m05_d26_h14_m22_s35_bare_keras_v2.reconstruction_network, which coordinates with my model path, and the "reconstruction_network.py " see work nomally.

[DavidWatkins]

Can you try to visualize the voxel grid directly? Perhaps using https://matplotlib.org/stable/gallery/mplot3d/voxels.html. I think the voxel grid you are getting is poorly formatted for some reason and it could be from the input or output data.

[qyp-robot]

ok,I will try it! thank yuu!

[qyp-robot]

Hi! I find the inputdata of CNN as show follow, I save it as “binvox_rw.write(partial_vox, open('/home/qyp/parcial_point_test0107/voxel_file/inputdata_cnn1.binvox', 'w'))”： also, I find the output data from CNN as show follow: what's more ,the pcd file is true, the problem happens in the inputdata from CNN, my input point cloud file which has aboout 4000 points, is it smaller than you used? after all, the voxed file size is 40mmX40mmX40mm？ or you have other shape object point cloud data ? now, I get it from gazebo, and what data set have singeled view point cloud data? because the camera is not locating with me now. very thanks! @DavidWatkins

[DavidWatkins]

It looks like the voxelization is incorrect. You should only have a voxel grid corresponding to the partial of the Rubbermaid container. The voxel grid dimensions are 40x40x40 but the voxel size is flexible corresponding to the size of the point cloud. Can you look at the point cloud as a pcd file in pcl_viewer? It looks like your point cloud may be malformed. Your voxel grid should look like the ones shown on this page: http://crlab.cs.columbia.edu/visualtactilegrasping/.

[qyp-robot]

thank you for your replaying, delayed this test fot your suggestion due to other things, I will do it now,thank you

[qyp-robot]

I am sorry to bother you again, hi, my point cloud file like this show below: I wonder 40X40X40 is larger than my point cloud, because cloud number is only 7660, how can I adjust the voxeled size? I try set the patch_size as 5, but the command line have bug, like this: how can i solve it? thank you very much!

[qyp-robot]

40X40X40 measurement unit is mm？

[qyp-robot]

I see code show: how many voxels along a single dimension of the voxel grid. I understand this meaning, when my point cloud number is 7660, a single dimension of the voxel grid should set how much?

[DavidWatkins]

The voxel resolution is flexible with the size of the point cloud. It is 40^3 but not given a particular unit measurement beforehand. The number of points does not change the number of voxels either. If multiple points are in the same voxel, the voxel is still set to 1.

[DavidWatkins]

Take a look at: curvox.pc_vox_utils.pc_to_binvox_for_shape_completion

[qyp-robot]

Take a look at: curvox.pc_vox_utils.pc_to_binvox_for_shape_completion

ok, I already take a look. now, I can confirm the point cloud fire(.pcd file) voxeled incorrect ,I save the partial_vox file which is input of cnn, as the follow;

I search some .binvox file like chair, the partial_vox file is not true. my point cloud file from gazebo; how can I solve? or ues YCB file of partial point to test？ thank you!

[qyp-robot]

hi,I find my code input has trouble,in my python window,I use the topic is depth like this "pc_scene_completion_client.complete_scene("depth")", but my point cloud topis is /camera/depth/color/points in ROS;

so, I find the pointCloud before filtering is 362128 data points in "rosrun pc_object_completion_cnn mesh_completion_server.py depth" window, which can not match my original point cloud number 7660 of inoput, like this picture:

[DavidWatkins]

You don't have to run this inside of ROS. You can just extract the code specific to the completion and validate it is working. As long as the point cloud is a numpy array of size (n, 3) it should process.

I am not sure why the point cloud sizes differ. Is there another ROS topic broadcasting point clouds of different sizes? Can you try to use RViz to visualize what is being published?

[qyp-robot]

hi, I wonder the goal.partial_cloud information that who supply in the picture below.

because I have already modified the , the"/camera/depth/color/points" is my pointcloud topic, I not ues the "pc_filter" cpp program; it can see from the picture:

I ues the static_transform_publisher of tf that publish the transform of camera_frame and word_frame;

thirdly, I run the "rosrun pc_object_completion_cnn mesh_completion_server.py depth" in bash window which is successful, lastly. I transfer command like follow picture:

my trouble exist who supply the information of goal.partial_cloud,which subscribe the topic /camera/depth/color/points? thank you for your reply

[qyp-robot]

in addition,I believe use the command "mesh_reconstruction

",I can get the output of CNN; but I can not generate the .binvox file, can not convert .pcd file to .binvox file,like read from file path, you can put some sample code for me ?

[qyp-robot]

thank you very much!

[qyp-robot]

hi! I am alredy voxeld successful! like this, I will to complete it ,thank you!

[DavidWatkins]

Good luck!

[qyp-robot]

Hi! @DavidWatkins doctor, I encounter some new difficulties，I find the output as the same with my original input .binvox file; which from mesh_reconstruction this command product mesh file by using .pcd file and .binvox file, it shows below:

[qyp-robot]

can you tell me what the problem might be？thank you!

[qyp-robot]

HI! @DavidWatkins ,now, I can not get the cnn output binvox file, I know mesh_reconstruction program need the output binvox file from CNN and original pointcloud file; above I use the original binvox file of original pointcloud,so the mesh is incorrect! so I modify the code, I want the input binvox file is my produced,like this: others is not changing, I think this can work normaly, but, the result still is original that as same as the picture, I show the output binvox from CNN like this:

addition, I use the virtual machine runs ubuntu, only use the cpu can run the cnn?

[DavidWatkins]

I am not sure. You appear to be doing things how they were written. I've never tested inside of a virtual machine, though. That may be causing issues.