Using PointNeXt as a new backbone

[Rolandxx7]

Hi,

Thanks for your great job! I have used PointNeXt as a new backbone to replace the backbone(PointNet++) in Point-RCNN, but I found the performance is very poor. Specifically, I have tried the following backbones: PointNeXt-s, PointNeXt-b and PointNeXt-l. Also, I have ruled out the causes of overfitting and underfitting, and the best Average Precision of the model using PointNeXt as backbone is about 10 points lower than before. Besides, I have checked my code for new backbone and found no problems.

So I really need your help, and hope that you can give me some advice!

[guochengqian]

@Rolandxx7 thanks for letting me know this. I think it is mostly because of the initial radius. Please reply to me with the following:

1. Are you able to reproduce point-rcnn with pointnet++, which as you mention, is 10 points better than pointnext?
2. the configuration of the PointNet++ backbone
3. the configuration of PointNeXt-S or -B

[Rolandxx7]

@Rolandxx7 thanks for letting me know this. I think it is mostly because of the initial radius. Please reply to me with the following:

1. Are you able to reproduce point-rcnn with pointnet++, which as you mention, is 10 points better than pointnext?
2. the configuration of the PointNet++ backbone
3. the configuration of PointNeXt-S or -B

1. The backbone of original PointRCNN is based on pointnet++. So what I did was replace pointnet++ with pointnext, and it is indeed 10 points worse than pointnet++.

2. the configuration of the PointNet++ backbone: ` # config of network structure BACKBONE: pointnet2_msg USE_BN: True NUM_POINTS: 16384

   SA_CONFIG: NPOINTS: [4096, 1024, 256, 64] RADIUS: [[0.1, 0.5], [0.5, 1.0], [1.0, 2.0], [2.0, 4.0]] NSAMPLE: [[16, 32], [16, 32], [16, 32], [16, 32]] MLPS: [[[16, 16, 32], [32, 32, 64]], [[64, 64, 128], [64, 96, 128]], [[128, 196, 256], [128, 196, 256]], [[256, 256, 512], [256, 384, 512]]] FP_MLPS: [[128, 128], [256, 256], [512, 512], [512, 512]] CLS_FC: [128] REG_FC: [128] DP_RATIO: 0.5`

3.I use the default configuration of PointNeXt-S and -B of 'cfgs/s3dis'.

[guochengqian]

okay. First, are you able to reproduce the original PointRCNN with PointNet++? This is the first thing you have to make sure. If you can reproduce PointRCNN, then, my suggestion is the following: use the default pointnext-s, but play with the radius. try to use the same radius as PointRCNN at first, then ablate the following values for initial radius: 0.1, 0.2, 0.25, 0.3

[Rolandxx7]

okay. First, are you able to reproduce the original PointRCNN with PointNet++? This is the first thing you have to make sure. If you can reproduce PointRCNN, then, my suggestion is the following: use the default pointnext-s, but play with the radius. try to use the same radius as PointRCNN at first, then ablate the following values for initial radius: 0.1, 0.2, 0.25, 0.3

@guochengqian Thanks for advice, and I will try it soon. I have reproduced the original PointRCNN with PointNet++. Also, I would like to emphasize that I found that the pointnext-s and pointnext-b can fits well on the training set after training enough epochs, but poorly on the validation set. Do you think this is also caused by the radius? Moreover, I have tried the default pointnext-s before, and does the default PointNext-s not play with the radius? I find that the initial radius of the default pointnext-s is also 0.1.

[guochengqian]

hi Roland, 3D dataset is super sensitive to the radius. It defines the receptive fields. An important reason for Overfitting is an improper radius.

PointRCNN has played with radius. if you find correctly, they multiply the radius by 5 times, not the regular 2 times. RADIUS: [[0.1, 0.5], [0.5, 1.0], [1.0, 2.0], [2.0, 4.0]]

in other experiments, people usually always double radius, like pointnet++, kpconv, and pointnext. Thats why I want you to to try the same radius as them, and play with radius by yourself as well.

[Rolandxx7]

hi Roland, 3D dataset is super sensitive to the radius. It defines the receptive fields. An important reason for Overfitting is an improper radius.

PointRCNN has played with radius. if you find correctly, they multiply the radius by 5 times, not the regular 2 times. RADIUS: [[0.1, 0.5], [0.5, 1.0], [1.0, 2.0], [2.0, 4.0]]

in other experiments, people usually always double radius, like pointnet++, kpconv, and pointnext. Thats why I want you to to try the same radius as them, and play with radius by yourself as well.

@guochengqian Thank you for your explanation! I would like to ask one more question.

As we can see: RADIUS in PointRCNN: [[0.1, 0.5], [0.5, 1.0], [1.0, 2.0], [2.0, 4.0]]. In the SA module of PointRCNN, it uses the method of Multi-scale grouping (MSG), so it has two radius values at each stage of SA modules. I would like to know if I want to play with the radius, do I need to implement that method of MSG in pointnext?

[guochengqian]

just use the vanilla version is fine.

[Rolandxx7]

@guochengqian Hi, I have used Pointnext-s and tried the scheme of changing the radius you told me, and the accuracy is indeed improved. However, it still does not exceed the point-rcnn based on PointNet++ backbone after I tried many combinations of radii. So I have two questions here:

1. For the default pointnext-s radius: radii:[[0.1], [0.1], [0.2], [0.4], [0.8]], is the first [0.1] useless?
2. Is there any way to adjust it next?

[mikkelmedm]

Hi Roland From my understanding the first [0.1] is not used, no, as the first layer doesn't make use of the radius, it is just a simple 1D convolution/MLP layer - the radii is only used in the following layers where the query and group methods are used.

[guochengqian]

@Rolandxx7 our code also support the multi-grouping version of pointnet++. but you might tune the code just a little bit to make the multi-grouping pointnext work.

[Rolandxx7]

@Rolandxx7 our code also support the multi-grouping version of pointnet++. but you might tune the code just a little bit to make the multi-grouping pointnext work.

@guochengqian Thank you for your reply! Could you please tell me the location of your multi-grouping, I haven't been able to find that.