Accuracy Drop by Point Order?

[dsim174]

Hi,

I try to follow the votenet code, but I have a question in line 192-194 of sunrgbd_detection_data.py.

I trained the model with the original code,

    point_cloud, choices = pc_util.random_sampling(point_cloud, self.num_points, return_choices=True)
    point_votes_mask = point_votes[choices,0]
    point_votes = point_votes[choices,1:]

Then, I evaluated the model with sorted points,

    _, choices = pc_util.random_sampling(point_cloud, self.num_points, return_choices=True)
    choices.sort()        
    point_votes_mask = point_votes[choices,0]
    point_votes = point_votes[choices,1:]
    point_cloud = point_cloud[choices]

But, I got only 32.6 mIoU (The original evaluation accuracy was 58.2 mIoU)

Why did accuracy drop? Did I mistake?

Or, is the votenet model not invariant in point order?

Thank you.

[tsakelliou]

Not a solution to your problem, but to say I had the same issue. By running some tests, I realized that, when doing inference, if a pointcloud is not shuffled/randomly sampled and has some structure, the results are much worse.

I am also very interested in hearing about the reason, as Pointnet++ is invariant in order.

[dsim174]

@tsakelliou My problem was caused by data augmentation. When I trained the model with sorted points, I closely got the original accuracy (57.9 mIoU). In my opinion, Pointnet++ is invariant in order if it is well trained.