search

JudasDie / SOTS

Single object tracking and segmentation.
478 stars 76 forks source link

testing question #74

Open wyl12 opened 1 year ago

wyl12 commented 1 year ago

/home/lsw/miniconda3/envs/SOTS/bin/python /home/lsw/SOT/SOTS/tracking/test_sot.py model backbone: ResNet50Dilated model neck: ShrinkChannelS3S4 model head: Learn2Match model build done! SiamInference( (backbone): ResNet50Dilated( (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), bias=False) (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False) (layer1): Sequential( (0): Bottleneck( (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) (downsample): Sequential( (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) ) (1): Bottleneck( (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) (2): Bottleneck( (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) ) (layer2): Sequential( (0): Bottleneck( (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), bias=False) (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) (downsample): Sequential( (0): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), bias=False) (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) ) (1): Bottleneck( (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) (2): Bottleneck( (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) (3): Bottleneck( (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) ) (layer3): Sequential( (0): Bottleneck( (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) (downsample): Sequential( (0): Conv2d(512, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) ) (1): Bottleneck( (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False) (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) (2): Bottleneck( (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False) (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) (3): Bottleneck( (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False) (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) (4): Bottleneck( (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False) (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) (5): Bottleneck( (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False) (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (relu): ReLU(inplace=True) ) ) ) (neck): ShrinkChannelS3S4( (downsample): Sequential( (0): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU(inplace=True) ) (downsample_s3): Sequential( (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU(inplace=True) ) ) (head): Learn2Match( (regression): L2Mregression( (reg_encode): SimpleMatrix( (matrix11_k): Sequential( (0): Conv2d(256, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) ) (matrix11_s): Sequential( (0): Conv2d(256, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) ) ) (roi_te): roi_template( (fea_encoder): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) (fea_encoder_s3): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) ) (spatial_conv): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) ) (spatial_conv_s3): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) ) ) (LTM): LTM( (FiLM): FiLM( (s_embed): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (conv_g): Sequential( (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (1): LeakyReLU(negative_slope=0.1) ) (conv_b): Sequential( (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (1): LeakyReLU(negative_slope=0.1) ) ) (PC): PairRelation( (s_embed): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1)) (t_embed): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1)) (down): Conv2d(225, 256, kernel_size=(1, 1), stride=(1, 1)) ) (embed2): Sequential( (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) ) ) (bbox_tower): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): ReLU() (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (5): ReLU() (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (8): ReLU() ) (bbox_pred): Conv2d(256, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (classification): L2Mclassification( (LTM): LTM( (Transformer): SimpleSelfAtt( (s_embed): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (t_embed_v): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (t_embed): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (trans): MultiheadAttention( (out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True) ) ) (FiLM): FiLM( (s_embed): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (conv_g): Sequential( (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (1): LeakyReLU(negative_slope=0.1) ) (conv_b): Sequential( (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1)) (1): LeakyReLU(negative_slope=0.1) ) ) (embed2): Sequential( (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) ) ) (roi_cls): roi_classification( (fea_encoder): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) (3): Conv2d(256, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (5): LeakyReLU(negative_slope=0.1) ) (fea_encoder_s3): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) (3): Conv2d(256, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (5): LeakyReLU(negative_slope=0.1) ) (down_spatial_conv): Sequential( (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1)) (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) ) (down_spatial_linear): Sequential( (0): Linear(in_features=128, out_features=128, bias=True) (1): LeakyReLU(negative_slope=0.1) ) (down_spatial_conv_s3): Sequential( (0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1)) (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) ) (down_target_s3): Sequential( (0): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1)) (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) ) (down_target_s4): Sequential( (0): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1)) (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) ) (down_spatial_linear_s3): Sequential( (0): Linear(in_features=128, out_features=128, bias=True) (1): LeakyReLU(negative_slope=0.1) ) (merge_s3s4_s2): Sequential( (0): Linear(in_features=256, out_features=256, bias=True) (1): LeakyReLU(negative_slope=0.1) (2): Linear(in_features=256, out_features=256, bias=True) (3): LeakyReLU(negative_slope=0.1) ) (merge_s3s4_s1): Sequential( (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (2): LeakyReLU(negative_slope=0.1) (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (5): LeakyReLU(negative_slope=0.1) ) (pred_s1): Conv2d(256, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (pred_s2): Linear(in_features=256, out_features=1, bias=True) ) ) ) ) ===> init Siamese <==== load pretrained model from ../snapshot/AutoMatch.pth remove prefix 'module.' change features.features to backbone 2022-12-02 22:47:17.534 | INFO | lib.utils.model_helper:check_keys:178 - missing keys:[] Traceback (most recent call last): File "/home/lsw/SOT/SOTS/tracking/test_sot.py", line 150, in main() File "/home/lsw/SOT/SOTS/tracking/test_sot.py", line 135, in main siam_net = loader.load_pretrain(siam_net, resume, addhead=True, print_unuse=False) File "/home/lsw/SOT/SOTS/tracking/../lib/utils/model_helper.py", line 209, in load_pretrain model.load_state_dict(pretrained_dict, strict=True) File "/home/lsw/miniconda3/envs/Unicorn/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1483, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for SiamInference: Unexpected key(s) in state_dict: "head.regression.LTM.GuidedSP.s_embed.weight", "head.regression.LTM.GuidedSP.s_embed.bias", "head.regression.LTM.GuidedSP.t_embed.weight", "head.regression.LTM.GuidedSP.t_embed.bias", "head.regression.LTM.PointDW.s_embed.weight", "head.regression.LTM.PointDW.s_embed.bias", "head.regression.LTM.PointDW.t_embed.weight", "head.regression.LTM.PointDW.t_embed.bias", "head.regression.LTM.PointAdd.s_embed.weight", "head.regression.LTM.PointAdd.s_embed.bias", "head.regression.LTM.PointAdd.t_embed.weight", "head.regression.LTM.PointAdd.t_embed.bias", "head.regression.LTM.Transformer.s_embed.weight", "head.regression.LTM.Transformer.s_embed.bias", "head.regression.LTM.Transformer.t_embed_v.weight", "head.regression.LTM.Transformer.t_embed_v.bias", "head.regression.LTM.Transformer.t_embed.weight", "head.regression.LTM.Transformer.t_embed.bias", "head.regression.LTM.Transformer.trans.in_proj_weight", "head.regression.LTM.Transformer.trans.in_proj_bias", "head.regression.LTM.Transformer.trans.out_proj.weight", "head.regression.LTM.Transformer.trans.out_proj.bias", "head.classification.LTM.GuidedSP.s_embed.weight", "head.classification.LTM.GuidedSP.s_embed.bias", "head.classification.LTM.GuidedSP.t_embed.weight", "head.classification.LTM.GuidedSP.t_embed.bias", "head.classification.LTM.PointDW.s_embed.weight", "head.classification.LTM.PointDW.s_embed.bias", "head.classification.LTM.PointDW.t_embed.weight", "head.classification.LTM.PointDW.t_embed.bias", "head.classification.LTM.PointAdd.s_embed.weight", "head.classification.LTM.PointAdd.s_embed.bias", "head.classification.LTM.PointAdd.t_embed.weight", "head.classification.LTM.PointAdd.t_embed.bias", "head.classification.LTM.PC.s_embed.weight", "head.classification.LTM.PC.s_embed.bias", "head.classification.LTM.PC.t_embed.weight", "head.classification.LTM.PC.t_embed.bias", "head.classification.LTM.PC.down.weight", "head.classification.LTM.PC.down.bias".

Process finished with exit code 1 图片

Hello,i met this problem when i try to testing ,but i can not solve it.Could you help me to solve it if you have some time?

faicaiwawa commented 1 year ago

I also met this problem,it seem model weight is not match with code .have you solve this problem?

YOLOCy commented 1 year ago

I also meet this question. May I ask if you have resolved it? If so, could you please let me know the method

Air1000thsummer commented 3 months ago

same issue . seams wrong pretrain.pth