Mismatched dimensions on custom data

[tejaswi-digumarti-oxb]

I am following the superpoint_transformer_tutorial.ipynb on a custom pointcloud following the kitti360 experiment.

I generate the Data object from a .pcd file using open3d as follows.

def read_point_cloud(filepath, xyz=True, rgb=True, intensity=True, semantic=True, max_intensity=600):
    data = Data()
    pcd = o3d.io.read_point_cloud(filepath)
    xyz_np = np.asarray(pcd.points)
    rgb_np = np.asarray(pcd.colors)
    if xyz:
        pos = torch.stack([torch.tensor(xyz_np[:, i]) for i in range(0, 3)], dim=-1)
        pos_offset = pos[0]
        data.pos = (pos - pos_offset).float()
        data.pos_offset = pos_offset
    if rgb:
        # rgb are float values in [0,1]
        data.rgb = to_float_rgb(torch.stack([torch.FloatTensor(rgb_np[:, i]) for i in range(0, 3)], dim=-1))
    if intensity:
        # intensity is stored in the red channel and is a float value in [0,1]
        data.intensity = torch.FloatTensor(rgb_np[:, 0])
    return data

The rest of the steps are the same, except that I replace dales with kitti360. In section 4.4 when I try to predict using the model, I get a dimension mismatch error

RuntimeError: mat1 and mat2 shapes cannot be multiplied (36781x15 and 12x32)

I see that SPT -> first_stage has Linear(in_features=12, out_features=32) when instantiated from cfg.model which explains the size of mat2. nag[0].x.shape returns torch.Size([36781, 11]), so where is the 15 in the shape of mat1 coming from? Am I missing something?

[drprojects]

Are you sure you modified the model and datamodule for KITTI-360 :

• the proper 'experiment/semantic/kitti360 config for both datamodule and model ?
• the proper KITTI-360-pretrained model from ?

[drprojects]

This is the second issue you open today.

If you ❤️ or use this project, don't forget to give it a ⭐, it means a lot to us !

[tejaswi-digumarti-oxb]

Yes, I am using the model from spt-2_kitti360.ckpt and the config as cfg = init_config(overrides=[f"experiment=semantic/kitti360.yaml"]).

[drprojects]

I investigated and indeed there was an issue when using KITTI-360. Can you please try again with the latest commit and let me know if this also runs on your end ?

[tejaswi-digumarti-oxb]

Yes, that works. Thanks for your prompt fix. Appreciate it 🙏