Mapping point-cloud to custom output

cisprague commented 5 years ago

Hi Erik,

I have several point clouds that represent different sub-maps of a terrain map. I don't need to perform segmentation or classification. Instead, I need to output the raw response of a submap. I then want to input this response into another neural network, which outputs a physically meaningful vector.

Could you suggest how to do this within Python? (not using the terminal) Suppose I have a tensor of 20 submaps, each having 1000 3D points (x.shape = (20, 1000, 3)).

erikwijmans commented 5 years ago

I am not sure what the "raw response of a submap" is.

cisprague commented 5 years ago

I have batches of pointclouds. Each batch is labelled with a 2x2 matrix of continuous real-numbers. I want to learn the mapping from a batch to its 2x2 matrix label of continuous real-numbers.

Input:

# 3 batches of pointclouds, each having 4153 points of 3 dimensions
# a.shape = torch.Size([3, 4153, 3])
x = tensor([[[ -4.4242, 179.5540,  -3.9953],
         [ -4.5037, 180.5730,  -3.9987],
         [ -4.5773, 181.5160,  -4.0026],
         ...,
         [  8.7374, 223.7580,  -4.0162],
         [  8.6664, 224.6600,  -3.9838],
         [  8.5804, 225.7520,  -4.0038]],

        [[ 12.6884, 180.4480,  -3.9466],
         [ 12.6097, 181.4470,  -3.9688],
         [ 12.5336, 182.4120,  -3.9716],
         ...,
         [ 26.1111, 225.3330,  -3.9838],
         [ 26.0312, 226.3350,  -3.9934],
         [ 25.9488, 227.3720,  -3.9833]],

        [[ 30.1386, 181.5060,  -3.9637],
         [ 30.0578, 182.5190,  -3.9860],
         [ 29.9806, 183.4880,  -3.9937],
         ...,
         [ 43.5112, 226.8030,  -4.0133],
         [ 43.4386, 227.7140,  -3.9794],
         [ 43.3492, 228.8230,  -4.0008]]], device='cuda:0')

Labels:

# 3 labels
# y.shape =torch.Size([3, 2, 2])
y = tensor([[[ 3.5590,  0.0000],
         [ 0.0000, 57.0456]],

        [[ 3.4989,  0.0000],
         [ 0.0000, 59.3808]],

        [[ 1.4000,  0.0000],
         [ 0.0000, 49.8878]]], device='cuda:0')

erikwijmans commented 5 years ago

You can remove the FC layer: https://github.com/erikwijmans/Pointnet2_PyTorch/blob/master/pointnet2/models/pointnet2_ssg_cls.py#L79 and then just output the features https://github.com/erikwijmans/Pointnet2_PyTorch/blob/master/pointnet2/models/pointnet2_ssg_cls.py#L112

cisprague commented 5 years ago

Ahh, I see, thank you!

erikwijmans / Pointnet2_PyTorch

Mapping point-cloud to custom output #51