yzqin
commented
1 year ago Hi @kimanton,
For point cloud the camera frame's Z axis points away from point cloud, this is the OpenGL/Computer Graphics Convention used by many software. We use blender to render depth so that it also follows this convention.
Now you have a OpenCV convention point cloud, i.e. all point have positive Z position. You can convert it to OpenGL convention with a rotation matrix before forwarding it to the s4g inference code. And convert the grasp pose back to the OpenCV frame convention if you want.
I think if the neural network is trained with one conventions, then it can hardly deal with the other one. I think the transformations are not a hard issue, it is some rotation matrix with only [-1, 0, 1] entries to convert from OpenGL convention to OpenCV convention.
kimanton
Hi again @yzqin I tried the test file you provided and noticed that camera frame's Z axis points away from point cloud - all points in point cloud have negative Z position In my case I have camera that points into point cloud - all point have positive Z position Now, it seems that the programm works fine in your case - when Z axis points away from point cloud - but its not doing the job when I am trying to feed it my point cloud with Z axis pointing into point cloud
I was wondering if I can tweak some parameters in your code to actually take in my point clouds, or if there is no way does that mean I just have to work out all required transformations?