Closed kxhit closed 5 years ago
I am not the author, but I experimented with a PointNet-like architecture and I visualized transformed point clouds. I was surprised to see that transformed point clouds are almost the same as original ones. And when I printed transformation matrices they indeed were very close to identity. So now I think that TNet on input have not much sense. In my experiments results were slightly better without TNet on input.
Hey @kuynzereb,
can you maybe explain me how the testing of the trained model works?
I mean during training an affine transformation is learned via the spatial transformer network for the points and features of every input point cloud. But how does this help and acctually work during testing the trained model? Does the model then try to transform the individual point clouds from the test set with one of the learned affine transformations in order to predict it?
Maybe you have an idea here.
Thanks in advance!
BG Roman
@kuynzereb Hi! I run the evaluate.py with the trained model (set parameter as default). Then I get the input_points(4x1024x3), transformed_points(4x1024x3) and TNet(4x3x3). I convert the input and the output to pcd file and visualize them with pcl_viewer.
without TNet1 eval mean loss: 0.777766 eval accuracy: 0.868912 eval avg class acc: 0.843538
Data and logs can be found here.
@romanskie During testing, the parameters are restored(the parameters are saved when model is trained) firstly. Then, the input_cloud_points(BatchxNx3) will be input to TNet and TNet will output a Batchx3x3 matrix. After that, the output of TNet will multiply with the input_cloud_points and get the transformed cloud points. Hope it helps you.
Hey @kxhit,
thanks for your response.
This is my understanding now :) The network learns two transformations (T1 and T2) for each point cloud of the traning set during training. Means like 5000 x 2 transformations if your train set is about 5000 point clouds? By learning these transformations, the trained model then knows how to transform the point clouds from the test set in order to predict them?
BG Roman
@romanskie Yeah, maybe the TNet learns to get information from the input points and then predicts a transformation matrix(3x3). Then, the matrix makes the input points rotate and scaled, which may be useful for the later feature extractions.
@kxhit
@romanskie I think @kxhit have answered your question :)
Thanks for the excellent work! The authors say, "We predict an affine transformation matrix by a mini-network and directly apply this transformation to the coordinates of input points." I'm curious what the point cloud looks like after the transformation. Is it a three-dimensional representation that a person can still understand?