Inference with Scannet validation and test dataset

[claragarciamoll]

Hello, first of all, congratulation on your fantastic work. Currently, I have the code working. First, I prepared the dataset running the scripts:

• python split_data.py
• python prepare_data_inst.py --data_split val
• python prepare_data_inst.py --data_split test Once I had the dataset in an appropriated order, when I ran the latest version (https://github.com/thangvubk/SoftGroup/commit/cf88d9be41ae83a70f9100856a3ca15ee4ddcee9) of the inference (test.py) with the recommended configuration, I obtain semantic_prediction equal to 1 (in every point) on each point and the pred_instances is empty (in both cases: validation set using labels and test set without labeling). As an extra information, the only part which is different in my environment is the Pytorch version, which is 1.10.2. How can I solve this problem?

[claragarciamoll]

I believe I have found the problem. Because some warnings appear when running test.py in the part of loading the model. I attach the error.

2022-09-07 14:09:50,842 - INFO - removed keys in source state_dict due to size mismatch: input_conv.0.weight, unet.blocks.block0.conv_branch.2.weight, unet.blocks.block0.conv_branch.5.weight, unet.blocks.block1.conv_branch.2.weight, unet.blocks.block1.conv_branch.5.weight, unet.conv.2.weight, unet.u.blocks.block0.conv_branch.2.weight, unet.u.blocks.block0.conv_branch.5.weight, unet.u.blocks.block1.conv_branch.2.weight, unet.u.blocks.block1.conv_branch.5.weight, unet.u.conv.2.weight, unet.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.conv.2.weight, unet.u.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.u.conv.2.weight, unet.u.u.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.u.u.conv.2.weight, unet.u.u.u.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.u.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.u.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.u.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.u.u.u.conv.2.weight, unet.u.u.u.u.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.u.u.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.u.u.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.u.u.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.u.u.u.deconv.2.weight, unet.u.u.u.u.u.blocks_tail.block0.i_branch.0.weight, unet.u.u.u.u.u.blocks_tail.block0.conv_branch.2.weight, unet.u.u.u.u.u.blocks_tail.block0.conv_branch.5.weight, unet.u.u.u.u.u.blocks_tail.block1.conv_branch.2.weight, unet.u.u.u.u.u.blocks_tail.block1.conv_branch.5.weight, unet.u.u.u.u.deconv.2.weight, unet.u.u.u.u.blocks_tail.block0.i_branch.0.weight, unet.u.u.u.u.blocks_tail.block0.conv_branch.2.weight, unet.u.u.u.u.blocks_tail.block0.conv_branch.5.weight, unet.u.u.u.u.blocks_tail.block1.conv_branch.2.weight, unet.u.u.u.u.blocks_tail.block1.conv_branch.5.weight, unet.u.u.u.deconv.2.weight, unet.u.u.u.blocks_tail.block0.i_branch.0.weight, unet.u.u.u.blocks_tail.block0.conv_branch.2.weight, unet.u.u.u.blocks_tail.block0.conv_branch.5.weight, unet.u.u.u.blocks_tail.block1.conv_branch.2.weight, unet.u.u.u.blocks_tail.block1.conv_branch.5.weight, unet.u.u.deconv.2.weight, unet.u.u.blocks_tail.block0.i_branch.0.weight, unet.u.u.blocks_tail.block0.conv_branch.2.weight, unet.u.u.blocks_tail.block0.conv_branch.5.weight, unet.u.u.blocks_tail.block1.conv_branch.2.weight, unet.u.u.blocks_tail.block1.conv_branch.5.weight, unet.u.deconv.2.weight, unet.u.blocks_tail.block0.i_branch.0.weight, unet.u.blocks_tail.block0.conv_branch.2.weight, unet.u.blocks_tail.block0.conv_branch.5.weight, unet.u.blocks_tail.block1.conv_branch.2.weight, unet.u.blocks_tail.block1.conv_branch.5.weight, unet.deconv.2.weight, unet.blocks_tail.block0.i_branch.0.weight, unet.blocks_tail.block0.conv_branch.2.weight, unet.blocks_tail.block0.conv_branch.5.weight, unet.blocks_tail.block1.conv_branch.2.weight, unet.blocks_tail.block1.conv_branch.5.weight 2022-09-07 14:09:50,843 - INFO - missing keys in source state_dict: input_conv.0.weight, unet.blocks.block0.conv_branch.2.weight, unet.blocks.block0.conv_branch.5.weight, unet.blocks.block1.conv_branch.2.weight, unet.blocks.block1.conv_branch.5.weight, unet.conv.2.weight, unet.u.blocks.block0.conv_branch.2.weight, unet.u.blocks.block0.conv_branch.5.weight, unet.u.blocks.block1.conv_branch.2.weight, unet.u.blocks.block1.conv_branch.5.weight, unet.u.conv.2.weight, unet.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.conv.2.weight, unet.u.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.u.conv.2.weight, unet.u.u.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.u.u.conv.2.weight, unet.u.u.u.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.u.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.u.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.u.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.u.u.u.conv.2.weight, unet.u.u.u.u.u.u.blocks.block0.conv_branch.2.weight, unet.u.u.u.u.u.u.blocks.block0.conv_branch.5.weight, unet.u.u.u.u.u.u.blocks.block1.conv_branch.2.weight, unet.u.u.u.u.u.u.blocks.block1.conv_branch.5.weight, unet.u.u.u.u.u.deconv.2.weight, unet.u.u.u.u.u.blocks_tail.block0.i_branch.0.weight, unet.u.u.u.u.u.blocks_tail.block0.conv_branch.2.weight, unet.u.u.u.u.u.blocks_tail.block0.conv_branch.5.weight, unet.u.u.u.u.u.blocks_tail.block1.conv_branch.2.weight, unet.u.u.u.u.u.blocks_tail.block1.conv_branch.5.weight, unet.u.u.u.u.deconv.2.weight, unet.u.u.u.u.blocks_tail.block0.i_branch.0.weight, unet.u.u.u.u.blocks_tail.block0.conv_branch.2.weight, unet.u.u.u.u.blocks_tail.block0.conv_branch.5.weight, unet.u.u.u.u.blocks_tail.block1.conv_branch.2.weight, unet.u.u.u.u.blocks_tail.block1.conv_branch.5.weight, unet.u.u.u.deconv.2.weight, unet.u.u.u.blocks_tail.block0.i_branch.0.weight, unet.u.u.u.blocks_tail.block0.conv_branch.2.weight, unet.u.u.u.blocks_tail.block0.conv_branch.5.weight, unet.u.u.u.blocks_tail.block1.conv_branch.2.weight, unet.u.u.u.blocks_tail.block1.conv_branch.5.weight, unet.u.u.deconv.2.weight, unet.u.u.blocks_tail.block0.i_branch.0.weight, unet.u.u.blocks_tail.block0.conv_branch.2.weight, unet.u.u.blocks_tail.block0.conv_branch.5.weight, unet.u.u.blocks_tail.block1.conv_branch.2.weight, unet.u.u.blocks_tail.block1.conv_branch.5.weight, unet.u.deconv.2.weight, unet.u.blocks_tail.block0.i_branch.0.weight, unet.u.blocks_tail.block0.conv_branch.2.weight, unet.u.blocks_tail.block0.conv_branch.5.weight, unet.u.blocks_tail.block1.conv_branch.2.weight, unet.u.blocks_tail.block1.conv_branch.5.weight, unet.deconv.2.weight, unet.blocks_tail.block0.i_branch.0.weight, unet.blocks_tail.block0.conv_branch.2.weight, unet.blocks_tail.block0.conv_branch.5.weight, unet.blocks_tail.block1.conv_branch.2.weight, unet.blocks_tail.block1.conv_branch.5.weight, tiny_unet.blocks.block0.conv_branch.0.weight, tiny_unet.blocks.block0.conv_branch.0.bias, tiny_unet.blocks.block0.conv_branch.0.running_mean, tiny_unet.blocks.block0.conv_branch.0.running_var, tiny_unet.blocks.block0.conv_branch.2.weight, tiny_unet.blocks.block0.conv_branch.3.weight, tiny_unet.blocks.block0.conv_branch.3.bias, tiny_unet.blocks.block0.conv_branch.3.running_mean, tiny_unet.blocks.block0.conv_branch.3.running_var, tiny_unet.blocks.block0.conv_branch.5.weight, tiny_unet.blocks.block1.conv_branch.0.weight, tiny_unet.blocks.block1.conv_branch.0.bias, tiny_unet.blocks.block1.conv_branch.0.running_mean, tiny_unet.blocks.block1.conv_branch.0.running_var, tiny_unet.blocks.block1.conv_branch.2.weight, tiny_unet.blocks.block1.conv_branch.3.weight, tiny_unet.blocks.block1.conv_branch.3.bias, tiny_unet.blocks.block1.conv_branch.3.running_mean, tiny_unet.blocks.block1.conv_branch.3.running_var, tiny_unet.blocks.block1.conv_branch.5.weight, tiny_unet.conv.0.weight, tiny_unet.conv.0.bias, tiny_unet.conv.0.running_mean, tiny_unet.conv.0.running_var, tiny_unet.conv.2.weight, tiny_unet.u.blocks.block0.conv_branch.0.weight, tiny_unet.u.blocks.block0.conv_branch.0.bias, tiny_unet.u.blocks.block0.conv_branch.0.running_mean, tiny_unet.u.blocks.block0.conv_branch.0.running_var, tiny_unet.u.blocks.block0.conv_branch.2.weight, tiny_unet.u.blocks.block0.conv_branch.3.weight, tiny_unet.u.blocks.block0.conv_branch.3.bias, tiny_unet.u.blocks.block0.conv_branch.3.running_mean, tiny_unet.u.blocks.block0.conv_branch.3.running_var, tiny_unet.u.blocks.block0.conv_branch.5.weight, tiny_unet.u.blocks.block1.conv_branch.0.weight, tiny_unet.u.blocks.block1.conv_branch.0.bias, tiny_unet.u.blocks.block1.conv_branch.0.running_mean, tiny_unet.u.blocks.block1.conv_branch.0.running_var, tiny_unet.u.blocks.block1.conv_branch.2.weight, tiny_unet.u.blocks.block1.conv_branch.3.weight, tiny_unet.u.blocks.block1.conv_branch.3.bias, tiny_unet.u.blocks.block1.conv_branch.3.running_mean, tiny_unet.u.blocks.block1.conv_branch.3.running_var, tiny_unet.u.blocks.block1.conv_branch.5.weight, tiny_unet.deconv.0.weight, tiny_unet.deconv.0.bias, tiny_unet.deconv.0.running_mean, tiny_unet.deconv.0.running_var, tiny_unet.deconv.2.weight, tiny_unet.blocks_tail.block0.i_branch.0.weight, tiny_unet.blocks_tail.block0.conv_branch.0.weight, tiny_unet.blocks_tail.block0.conv_branch.0.bias, tiny_unet.blocks_tail.block0.conv_branch.0.running_mean, tiny_unet.blocks_tail.block0.conv_branch.0.running_var, tiny_unet.blocks_tail.block0.conv_branch.2.weight, tiny_unet.blocks_tail.block0.conv_branch.3.weight, tiny_unet.blocks_tail.block0.conv_branch.3.bias, tiny_unet.blocks_tail.block0.conv_branch.3.running_mean, tiny_unet.blocks_tail.block0.conv_branch.3.running_var, tiny_unet.blocks_tail.block0.conv_branch.5.weight, tiny_unet.blocks_tail.block1.conv_branch.0.weight, tiny_unet.blocks_tail.block1.conv_branch.0.bias, tiny_unet.blocks_tail.block1.conv_branch.0.running_mean, tiny_unet.blocks_tail.block1.conv_branch.0.running_var, tiny_unet.blocks_tail.block1.conv_branch.2.weight, tiny_unet.blocks_tail.block1.conv_branch.3.weight, tiny_unet.blocks_tail.block1.conv_branch.3.bias, tiny_unet.blocks_tail.block1.conv_branch.3.running_mean, tiny_unet.blocks_tail.block1.conv_branch.3.running_var, tiny_unet.blocks_tail.block1.conv_branch.5.weight, tiny_unet_outputlayer.0.weight, tiny_unet_outputlayer.0.bias, tiny_unet_outputlayer.0.running_mean, tiny_unet_outputlayer.0.running_var, iou_score_linear.weight, iou_score_linear.bias 2022-09-07 14:09:50,843 - INFO - unexpected key in source state_dict: intra_ins_unet.blocks.block0.conv_branch.0.weight, intra_ins_unet.blocks.block0.conv_branch.0.bias, intra_ins_unet.blocks.block0.conv_branch.0.running_mean, intra_ins_unet.blocks.block0.conv_branch.0.running_var, intra_ins_unet.blocks.block0.conv_branch.0.num_batches_tracked, intra_ins_unet.blocks.block0.conv_branch.2.weight, intra_ins_unet.blocks.block0.conv_branch.3.weight, intra_ins_unet.blocks.block0.conv_branch.3.bias, intra_ins_unet.blocks.block0.conv_branch.3.running_mean, intra_ins_unet.blocks.block0.conv_branch.3.running_var, intra_ins_unet.blocks.block0.conv_branch.3.num_batches_tracked, intra_ins_unet.blocks.block0.conv_branch.5.weight, intra_ins_unet.blocks.block1.conv_branch.0.weight, intra_ins_unet.blocks.block1.conv_branch.0.bias, intra_ins_unet.blocks.block1.conv_branch.0.running_mean, intra_ins_unet.blocks.block1.conv_branch.0.running_var, intra_ins_unet.blocks.block1.conv_branch.0.num_batches_tracked, intra_ins_unet.blocks.block1.conv_branch.2.weight, intra_ins_unet.blocks.block1.conv_branch.3.weight, intra_ins_unet.blocks.block1.conv_branch.3.bias, intra_ins_unet.blocks.block1.conv_branch.3.running_mean, intra_ins_unet.blocks.block1.conv_branch.3.running_var, intra_ins_unet.blocks.block1.conv_branch.3.num_batches_tracked, intra_ins_unet.blocks.block1.conv_branch.5.weight, intra_ins_unet.conv.0.weight, intra_ins_unet.conv.0.bias, intra_ins_unet.conv.0.running_mean, intra_ins_unet.conv.0.running_var, intra_ins_unet.conv.0.num_batches_tracked, intra_ins_unet.conv.2.weight, intra_ins_unet.u.blocks.block0.conv_branch.0.weight, intra_ins_unet.u.blocks.block0.conv_branch.0.bias, intra_ins_unet.u.blocks.block0.conv_branch.0.running_mean, intra_ins_unet.u.blocks.block0.conv_branch.0.running_var, intra_ins_unet.u.blocks.block0.conv_branch.0.num_batches_tracked, intra_ins_unet.u.blocks.block0.conv_branch.2.weight, intra_ins_unet.u.blocks.block0.conv_branch.3.weight, intra_ins_unet.u.blocks.block0.conv_branch.3.bias, intra_ins_unet.u.blocks.block0.conv_branch.3.running_mean, intra_ins_unet.u.blocks.block0.conv_branch.3.running_var, intra_ins_unet.u.blocks.block0.conv_branch.3.num_batches_tracked, intra_ins_unet.u.blocks.block0.conv_branch.5.weight, intra_ins_unet.u.blocks.block1.conv_branch.0.weight, intra_ins_unet.u.blocks.block1.conv_branch.0.bias, intra_ins_unet.u.blocks.block1.conv_branch.0.running_mean, intra_ins_unet.u.blocks.block1.conv_branch.0.running_var, intra_ins_unet.u.blocks.block1.conv_branch.0.num_batches_tracked, intra_ins_unet.u.blocks.block1.conv_branch.2.weight, intra_ins_unet.u.blocks.block1.conv_branch.3.weight, intra_ins_unet.u.blocks.block1.conv_branch.3.bias, intra_ins_unet.u.blocks.block1.conv_branch.3.running_mean, intra_ins_unet.u.blocks.block1.conv_branch.3.running_var, intra_ins_unet.u.blocks.block1.conv_branch.3.num_batches_tracked, intra_ins_unet.u.blocks.block1.conv_branch.5.weight, intra_ins_unet.deconv.0.weight, intra_ins_unet.deconv.0.bias, intra_ins_unet.deconv.0.running_mean, intra_ins_unet.deconv.0.running_var, intra_ins_unet.deconv.0.num_batches_tracked, intra_ins_unet.deconv.2.weight, intra_ins_unet.blocks_tail.block0.i_branch.0.weight, intra_ins_unet.blocks_tail.block0.conv_branch.0.weight, intra_ins_unet.blocks_tail.block0.conv_branch.0.bias, intra_ins_unet.blocks_tail.block0.conv_branch.0.running_mean, intra_ins_unet.blocks_tail.block0.conv_branch.0.running_var, intra_ins_unet.blocks_tail.block0.conv_branch.0.num_batches_tracked, intra_ins_unet.blocks_tail.block0.conv_branch.2.weight, intra_ins_unet.blocks_tail.block0.conv_branch.3.weight, intra_ins_unet.blocks_tail.block0.conv_branch.3.bias, intra_ins_unet.blocks_tail.block0.conv_branch.3.running_mean, intra_ins_unet.blocks_tail.block0.conv_branch.3.running_var, intra_ins_unet.blocks_tail.block0.conv_branch.3.num_batches_tracked, intra_ins_unet.blocks_tail.block0.conv_branch.5.weight, intra_ins_unet.blocks_tail.block1.conv_branch.0.weight, intra_ins_unet.blocks_tail.block1.conv_branch.0.bias, intra_ins_unet.blocks_tail.block1.conv_branch.0.running_mean, intra_ins_unet.blocks_tail.block1.conv_branch.0.running_var, intra_ins_unet.blocks_tail.block1.conv_branch.0.num_batches_tracked, intra_ins_unet.blocks_tail.block1.conv_branch.2.weight, intra_ins_unet.blocks_tail.block1.conv_branch.3.weight, intra_ins_unet.blocks_tail.block1.conv_branch.3.bias, intra_ins_unet.blocks_tail.block1.conv_branch.3.running_mean, intra_ins_unet.blocks_tail.block1.conv_branch.3.running_var, intra_ins_unet.blocks_tail.block1.conv_branch.3.num_batches_tracked, intra_ins_unet.blocks_tail.block1.conv_branch.5.weight, intra_ins_outputlayer.0.weight, intra_ins_outputlayer.0.bias, intra_ins_outputlayer.0.running_mean, intra_ins_outputlayer.0.running_var, intra_ins_outputlayer.0.num_batches_tracked, score_linear.weight, score_linear.bias

Could you help me?

[thangvubk]

are you using correct checkpoint. it should be softgroup_scannet_spconv2.pth

[claragarciamoll]

Yes, I download them from this link https://drive.google.com/file/d/1XUNRfred9QAEUY__VdmSgZxGQ7peG5ms/view

[claragarciamoll]

Assuming than those warnings are not the problem and the pointcloud are preprocessed (as needed). Why the inference is not working (semantic_prediction equal to 1 (in every point) on each point and the pred_instances is empty)?

[thangvubk]

Could you check your inference command again? If the checkpoint is correct, it should have no warnings, otherwise, the prediction will be not correct.

[claragarciamoll]

Hi, I download again the weights and I try again the inference and now, it is working. So, the problem is solved. Thanks for your help.