search

VISION-SJTU / USOT

[ICCV2021] Learning to Track Objects from Unlabeled Videos
63 stars 7 forks source link

error while trying to train #12

Closed nadavmisgav closed 1 year ago

nadavmisgav commented 2 years ago

Trying to train on a custom dataset, named it got10k to change as little as I can in the code and config, I ran the preprocess stated in the README and received the following flow json got10k_flow.txt.

When I try to run the train using 

python -u ./scripts/train_usot.py --cfg experiments/train/USOT.yaml --gpus 0 --workers 8

I get the following error,

/content/drive/MyDrive/USOT/USOT-main
=> creating var/log/USOT
=> creating var/log/USOT/USOT_2022-08-12-13-57
Namespace(cfg='experiments/train/USOT.yaml', gpus='0', workers=8)
{'CHECKPOINT_DIR': 'var/snapshot',
 'GPUS': '0',
 'OUTPUT_DIR': 'var/log',
 'PRINT_FREQ': 10,
 'USOT': {'DATASET': {'FAR_SAMPLE': 3,
                      'GOT10K': {'ANNOTATION': '/content/drive/MyDrive/USOT/USOT-main/preprocessing/datasets_train/got10k/got10k_flow.json',
                                 'PATH': '/content/drive/MyDrive/USOT/USOT-main/dataset',
                                 'USE': 19000},
                      'LASOT': {'ANNOTATION': '/home/jlzheng/dataset/lasot_flow/train.json',
                                'PATH': '/home/jlzheng/dataset/lasot_flow/crop511/',
                                'USE': 13000},
                      'SCALE': 0.05,
                      'SCALEm': 0.18,
                      'SCALEs': 0.18,
                      'SHIFT': 4,
                      'SHIFTm': 64,
                      'SHIFTs': 64,
                      'VID': {'ANNOTATION': '/home/jlzheng/dataset/VID_flow/train.json',
                              'PATH': '/home/jlzheng/dataset/VID_flow/crop511/',
                              'USE': 14000},
                      'VIDEO_QUALITY': 0.4,
                      'YTVOS': {'ANNOTATION': '/home/jlzheng/dataset/ytvos_flow/train.json',
                                'PATH': '/home/jlzheng/dataset/ytvos_flow/crop511/',
                                'USE': 4000}},
          'TEST': {'DATA': 'GOT10K',
                   'END_EPOCH': 30,
                   'ISTRUE': False,
                   'MODEL': 'USOT',
                   'START_EPOCH': 10,
                   'THREADS': 11},
          'TRAIN': {'BASE_LR': 0.005,
                    'BATCH': 12,
                    'BATCH_STAGE_2': 12,
                    'CLS_RATIOS': [0.6, 0.5, 0.4],
                    'CLS_RATIO_SHIFT_EPOCHS': [0, 7, 10],
                    'END_EPOCH': 30,
                    'ISTRUE': True,
                    'LAMBDA_1_LIST': [0.3, 0.275, 0.25],
                    'LAMBDA_1_NAIVE': 0.2,
                    'LAMBDA_SHIFT_EPOCHS': [0, 7, 9],
                    'LAMBDA_TOTAL': 0.9,
                    'LAYERS_LR': 0.1,
                    'LR': {'KWARGS': {'end_lr': 2e-05, 'start_lr': 0.005},
                           'TYPE': 'log'},
                    'MEMORY_EPOCH': 6,
                    'MEMORY_NUM': 4,
                    'MODEL': 'USOT',
                    'MOMENTUM': 0.9,
                    'PRETRAIN': 'imagenet_pretrain.model',
                    'RESUME': False,
                    'SEARCH_SIZE': 255,
                    'START_EPOCH': 1,
                    'STRIDE': 8,
                    'TEMPLATE_SIZE': 127,
                    'TRAINABLE_LAYER': ['layer1', 'layer2', 'layer3'],
                    'UNFIX_EPOCH': 10,
                    'UNFIX_POLICY': 'log',
                    'WARMUP': {'EPOCH': 5,
                               'IFNOT': True,
                               'KWARGS': {'end_lr': 0.005,
                                          'start_lr': 0.0025,
                                          'step': 1},
                               'TYPE': 'step'},
                    'WARM_POLICY': 'step',
                    'WEIGHT_DECAY': 0.0001,
                    'WHICH_USE': ['GOT10K']}},
 'WORKERS': 8}
USOT(
  (criterion): BCEWithLogitsLoss()
  (features): ResNet50(
    (features): ResNet_plus2(
      (conv1): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), bias=False)
      (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (relu): ReLU(inplace)
      (maxpool): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
      (layer1): Sequential(
        (0): Bottleneck(
          (conv1): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
          (downsample): Sequential(
            (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
            (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          )
        )
        (1): Bottleneck(
          (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
        (2): Bottleneck(
          (conv1): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
      )
      (layer2): Sequential(
        (0): Bottleneck(
          (conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), bias=False)
          (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
          (downsample): Sequential(
            (0): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), bias=False)
            (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          )
        )
        (1): Bottleneck(
          (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
        (2): Bottleneck(
          (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
        (3): Bottleneck(
          (conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
      )
      (layer3): Sequential(
        (0): Bottleneck(
          (conv1): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
          (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
          (downsample): Sequential(
            (0): Conv2d(512, 1024, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
            (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          )
        )
        (1): Bottleneck(
          (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
          (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
        (2): Bottleneck(
          (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
          (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
        (3): Bottleneck(
          (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
          (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
        (4): Bottleneck(
          (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
          (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
        (5): Bottleneck(
          (conv1): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(2, 2), dilation=(2, 2), bias=False)
          (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (conv3): Conv2d(256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
          (bn3): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
          (relu): ReLU(inplace)
        )
      )
    )
  )
  (neck): AdjustLayer(
    (downsample): Sequential(
      (0): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
      (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    )
    (prpooling): PrRoIPool2D(kernel_size=(7, 7), spatial_scale=1.0)
  )
  (connect_model): box_tower_reg(
    (cls_encode): matrix(
      (matrix11_k): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix11_s): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix12_k): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), dilation=(2, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix12_s): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), dilation=(2, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix21_k): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), dilation=(1, 2), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix21_s): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), dilation=(1, 2), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
    )
    (reg_encode): matrix(
      (matrix11_k): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix11_s): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix12_k): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), dilation=(2, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix12_s): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), dilation=(2, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix21_k): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), dilation=(1, 2), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (matrix21_s): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), dilation=(1, 2), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
    )
    (cls_dw): GroupDW()
    (reg_dw): GroupDW()
    (conf_fusion): Conf_Fusion(
      (conf_gen): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
      (value_gen): Sequential(
        (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace)
      )
    )
    (bbox_tower): Sequential(
      (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU()
      (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU()
      (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (8): ReLU()
      (9): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (10): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (11): ReLU()
    )
    (cls_tower): Sequential(
      (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU()
      (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU()
      (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (8): ReLU()
      (9): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (10): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (11): ReLU()
    )
    (cls_memory_tower): Sequential(
      (0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): ReLU()
      (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): ReLU()
      (6): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (7): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (8): ReLU()
      (9): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (10): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (11): ReLU()
    )
    (bbox_pred): Conv2d(256, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (cls_pred): Conv2d(256, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (cls_memory_pred): Conv2d(256, 1, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  )
)
load pretrained model from ./pretrain/imagenet_pretrain.model
remove prefix 'module.'
remove prefix 'feature_extractor.'
missing keys:['connect_model.reg_encode.matrix11_k.1.bias', 'connect_model.cls_encode.matrix21_k.1.running_var', 'connect_model.bbox_tower.10.bias', 'connect_model.cls_encode.matrix12_s.0.weight', 'connect_model.bbox_tower.1.running_var', 'connect_model.bbox_tower.7.running_var', 'connect_model.cls_memory_pred.bias', 'connect_model.cls_tower.7.weight', 'connect_model.bbox_tower.3.bias', 'connect_model.bias', 'connect_model.cls_memory_tower.6.weight', 'connect_model.cls_memory_tower.7.running_mean', 'connect_model.cls_encode.matrix11_s.0.weight', 'connect_model.cls_encode.matrix11_s.1.bias', 'connect_model.bbox_tower.4.bias', 'connect_model.bbox_tower.9.bias', 'connect_model.bbox_tower.4.running_var', 'connect_model.conf_fusion.value_gen.1.weight', 'connect_model.cls_encode.matrix21_k.0.weight', 'connect_model.conf_fusion.conf_gen.0.weight', 'connect_model.bbox_tower.7.running_mean', 'connect_model.bbox_tower.0.bias', 'connect_model.reg_encode.matrix21_k.0.weight', 'connect_model.cls_tower.1.running_mean', 'connect_model.reg_encode.matrix11_s.1.running_var', 'connect_model.reg_encode.matrix12_k.1.weight', 'connect_model.reg_encode.matrix11_s.1.bias', 'connect_model.bbox_tower.9.weight', 'connect_model.cls_tower.1.bias', 'connect_model.bbox_tower.6.weight', 'connect_model.cls_encode.matrix12_k.1.running_var', 'neck.downsample.1.running_var', 'connect_model.cls_encode.matrix21_k.1.bias', 'connect_model.reg_encode.matrix21_s.1.weight', 'connect_model.conf_fusion.value_gen.1.running_mean', 'connect_model.cls_encode.matrix21_s.1.bias', 'connect_model.cls_encode.matrix12_k.1.running_mean', 'connect_model.cls_tower.10.running_var', 'connect_model.cls_memory_tower.3.weight', 'connect_model.cls_memory_tower.4.weight', 'connect_model.conf_fusion.value_gen.0.weight', 'connect_model.bbox_tower.0.weight', 'connect_model.cls_memory_tower.4.running_var', 'connect_model.cls_encode.matrix12_k.1.weight', 'neck.downsample.1.bias', 'connect_model.cls_encode.matrix12_s.1.running_mean', 'connect_model.conf_fusion.conf_gen.1.running_mean', 'connect_model.cls_encode.matrix11_k.0.weight', 'connect_model.reg_encode.matrix12_k.1.running_var', 'connect_model.cls_tower.0.weight', 'connect_model.reg_dw.weight', 'connect_model.cls_encode.matrix12_k.0.weight', 'connect_model.reg_encode.matrix11_k.1.weight', 'connect_model.cls_encode.matrix21_s.1.running_var', 'connect_model.conf_fusion.value_gen.1.bias', 'connect_model.cls_tower.9.weight', 'connect_model.reg_encode.matrix12_s.1.running_mean', 'connect_model.reg_encode.matrix21_k.1.weight', 'connect_model.reg_encode.matrix11_s.1.running_mean', 'connect_model.cls_encode.matrix11_s.1.running_var', 'connect_model.cls_tower.9.bias', 'connect_model.bbox_pred.weight', 'connect_model.cls_encode.matrix11_k.1.bias', 'connect_model.cls_tower.3.bias', 'neck.downsample.0.weight', 'connect_model.bbox_tower.1.running_mean', 'connect_model.reg_encode.matrix12_k.1.bias', 'connect_model.reg_encode.matrix12_s.0.weight', 'connect_model.bbox_tower.1.weight', 'connect_model.bbox_tower.4.weight', 'connect_model.cls_tower.7.running_var', 'connect_model.conf_fusion.conf_gen.0.bias', 'connect_model.cls_encode.matrix21_s.1.weight', 'connect_model.cls_pred.bias', 'connect_model.cls_dw.weight', 'connect_model.adjust', 'connect_model.reg_encode.matrix21_k.1.bias', 'connect_model.conf_fusion.value_gen.1.running_var', 'connect_model.cls_memory_tower.1.bias', 'connect_model.reg_encode.matrix21_s.1.running_mean', 'connect_model.cls_encode.matrix21_s.0.weight', 'connect_model.conf_fusion.conf_gen.1.running_var', 'connect_model.cls_memory_tower.0.bias', 'connect_model.cls_tower.10.bias', 'connect_model.bbox_tower.10.running_var', 'connect_model.cls_memory_tower.9.bias', 'connect_model.cls_encode.matrix11_s.1.weight', 'connect_model.reg_encode.matrix12_s.1.weight', 'connect_model.reg_encode.matrix12_s.1.running_var', 'connect_model.cls_encode.matrix12_s.1.bias', 'connect_model.cls_pred.weight', 'connect_model.cls_memory_tower.7.weight', 'connect_model.cls_tower.4.weight', 'connect_model.cls_memory_tower.6.bias', 'connect_model.reg_encode.matrix11_s.0.weight', 'connect_model.bbox_tower.6.bias', 'connect_model.cls_memory_tower.4.bias', 'connect_model.cls_tower.6.bias', 'connect_model.cls_memory_tower.0.weight', 'connect_model.cls_tower.4.running_mean', 'connect_model.cls_encode.matrix21_k.1.running_mean', 'connect_model.cls_encode.matrix12_s.1.running_var', 'connect_model.cls_tower.6.weight', 'connect_model.cls_tower.4.running_var', 'connect_model.cls_memory_tower.7.bias', 'connect_model.cls_tower.1.running_var', 'connect_model.cls_tower.7.running_mean', 'neck.downsample.1.running_mean', 'connect_model.cls_tower.0.bias', 'connect_model.cls_memory_pred.weight', 'connect_model.cls_encode.matrix21_s.1.running_mean', 'connect_model.bbox_tower.4.running_mean', 'connect_model.cls_tower.10.running_mean', 'connect_model.cls_tower.10.weight', 'connect_model.conf_fusion.conf_gen.1.weight', 'connect_model.cls_tower.3.weight', 'connect_model.cls_memory_tower.10.weight', 'connect_model.cls_encode.matrix11_k.1.weight', 'connect_model.cls_memory_tower.1.running_mean', 'connect_model.bbox_tower.10.running_mean', 'connect_model.bbox_pred.bias', 'connect_model.reg_encode.matrix11_k.1.running_mean', 'connect_model.bbox_tower.7.bias', 'connect_model.cls_memory_tower.3.bias', 'connect_model.bbox_tower.1.bias', 'connect_model.reg_encode.matrix21_k.1.running_var', 'connect_model.cls_memory_tower.9.weight', 'connect_model.reg_encode.matrix12_k.1.running_mean', 'connect_model.conf_fusion.value_gen.0.bias', 'connect_model.reg_encode.matrix11_k.0.weight', 'connect_model.bbox_tower.7.weight', 'connect_model.reg_encode.matrix12_s.1.bias', 'connect_model.cls_memory_tower.10.bias', 'connect_model.reg_encode.matrix12_k.0.weight', 'connect_model.cls_encode.matrix11_k.1.running_var', 'connect_model.cls_tower.4.bias', 'connect_model.cls_encode.matrix11_k.1.running_mean', 'connect_model.reg_encode.matrix21_s.1.running_var', 'connect_model.cls_tower.1.weight', 'connect_model.cls_encode.matrix12_k.1.bias', 'neck.downsample.1.weight', 'connect_model.cls_encode.matrix11_s.1.running_mean', 'connect_model.reg_encode.matrix11_s.1.weight', 'connect_model.conf_fusion.conf_gen.1.bias', 'connect_model.reg_encode.matrix21_s.0.weight', 'connect_model.cls_memory_tower.10.running_var', 'connect_model.cls_memory_tower.1.weight', 'connect_model.reg_encode.matrix21_s.1.bias', 'connect_model.reg_encode.matrix11_k.1.running_var', 'connect_model.reg_encode.matrix21_k.1.running_mean', 'connect_model.cls_memory_tower.4.running_mean', 'connect_model.cls_memory_tower.10.running_mean', 'connect_model.cls_encode.matrix21_k.1.weight', 'connect_model.cls_tower.7.bias', 'connect_model.cls_memory_tower.1.running_var', 'connect_model.bbox_tower.3.weight', 'connect_model.bbox_tower.10.weight', 'connect_model.cls_memory_tower.7.running_var', 'connect_model.cls_encode.matrix12_s.1.weight']
unused checkpoint keys:['features.features.layer4.0.bn2.bias', 'features.features.layer4.0.bn2.weight', 'features.features.layer4.1.bn1.bias', 'features.features.layer4.2.bn2.weight', 'features.features.layer4.0.conv1.weight', 'features.features.layer4.0.bn2.running_mean', 'features.features.layer4.2.conv2.weight', 'features.features.layer4.2.bn1.running_mean', 'features.features.layer4.0.bn1.weight', 'features.features.layer4.1.bn1.weight', 'features.features.layer4.2.bn3.bias', 'features.features.layer4.2.bn3.running_mean', 'features.features.layer4.1.bn2.running_var', 'features.features.layer4.0.downsample.1.running_var', 'features.features.layer4.0.bn3.running_mean', 'features.features.layer4.0.bn3.bias', 'features.features.layer4.2.bn3.weight', 'features.features.layer4.2.bn1.running_var', 'features.features.layer4.1.conv3.weight', 'features.features.layer4.1.bn1.running_var', 'features.features.layer4.1.conv2.weight', 'features.features.layer4.1.bn2.weight', 'features.features.layer4.1.bn1.running_mean', 'features.features.layer4.1.bn3.running_var', 'features.features.layer4.0.downsample.1.bias', 'features.features.layer4.2.conv3.weight', 'features.features.layer4.2.bn2.running_var', 'features.features.layer4.0.bn3.weight', 'features.features.layer4.0.bn2.running_var', 'features.features.layer4.2.bn2.running_mean', 'features.features.layer4.1.bn2.running_mean', 'features.features.layer4.1.bn3.weight', 'features.features.layer4.0.downsample.0.weight', 'features.features.layer4.0.bn3.running_var', 'features.features.layer4.1.bn3.bias', 'features.features.layer4.1.bn2.bias', 'features.features.layer4.2.bn1.bias', 'features.features.layer4.0.conv2.weight', 'features.features.layer4.2.bn2.bias', 'features.features.layer4.0.bn1.running_mean', 'features.features.layer4.0.bn1.bias', 'features.features.layer4.0.bn1.running_var', 'features.features.layer4.2.bn1.weight', 'features.features.layer4.2.conv1.weight', 'features.features.layer4.0.conv3.weight', 'features.features.layer4.0.downsample.1.running_mean', 'features.features.layer4.1.conv1.weight', 'features.features.layer4.0.downsample.1.weight', 'features.features.layer4.2.bn3.running_var', 'features.features.layer4.1.bn3.running_mean']
==========first check trainable==========
{'params': <filter object at 0x7fab08023050>, 'lr': 0.0005}
{'params': <generator object Module.parameters at 0x7fab080517d0>, 'lr': 0.005}
{'params': <generator object Module.parameters at 0x7fab080519d0>, 'lr': 0.005}
==========double check trainable==========
trainable params:
neck.downsample.0.weight
neck.downsample.1.weight
neck.downsample.1.bias
connect_model.adjust
connect_model.bias
connect_model.cls_encode.matrix11_k.0.weight
connect_model.cls_encode.matrix11_k.1.weight
connect_model.cls_encode.matrix11_k.1.bias
connect_model.cls_encode.matrix11_s.0.weight
connect_model.cls_encode.matrix11_s.1.weight
connect_model.cls_encode.matrix11_s.1.bias
connect_model.cls_encode.matrix12_k.0.weight
connect_model.cls_encode.matrix12_k.1.weight
connect_model.cls_encode.matrix12_k.1.bias
connect_model.cls_encode.matrix12_s.0.weight
connect_model.cls_encode.matrix12_s.1.weight
connect_model.cls_encode.matrix12_s.1.bias
connect_model.cls_encode.matrix21_k.0.weight
connect_model.cls_encode.matrix21_k.1.weight
connect_model.cls_encode.matrix21_k.1.bias
connect_model.cls_encode.matrix21_s.0.weight
connect_model.cls_encode.matrix21_s.1.weight
connect_model.cls_encode.matrix21_s.1.bias
connect_model.reg_encode.matrix11_k.0.weight
connect_model.reg_encode.matrix11_k.1.weight
connect_model.reg_encode.matrix11_k.1.bias
connect_model.reg_encode.matrix11_s.0.weight
connect_model.reg_encode.matrix11_s.1.weight
connect_model.reg_encode.matrix11_s.1.bias
connect_model.reg_encode.matrix12_k.0.weight
connect_model.reg_encode.matrix12_k.1.weight
connect_model.reg_encode.matrix12_k.1.bias
connect_model.reg_encode.matrix12_s.0.weight
connect_model.reg_encode.matrix12_s.1.weight
connect_model.reg_encode.matrix12_s.1.bias
connect_model.reg_encode.matrix21_k.0.weight
connect_model.reg_encode.matrix21_k.1.weight
connect_model.reg_encode.matrix21_k.1.bias
connect_model.reg_encode.matrix21_s.0.weight
connect_model.reg_encode.matrix21_s.1.weight
connect_model.reg_encode.matrix21_s.1.bias
connect_model.cls_dw.weight
connect_model.reg_dw.weight
connect_model.conf_fusion.conf_gen.0.weight
connect_model.conf_fusion.conf_gen.0.bias
connect_model.conf_fusion.conf_gen.1.weight
connect_model.conf_fusion.conf_gen.1.bias
connect_model.conf_fusion.value_gen.0.weight
connect_model.conf_fusion.value_gen.0.bias
connect_model.conf_fusion.value_gen.1.weight
connect_model.conf_fusion.value_gen.1.bias
connect_model.bbox_tower.0.weight
connect_model.bbox_tower.0.bias
connect_model.bbox_tower.1.weight
connect_model.bbox_tower.1.bias
connect_model.bbox_tower.3.weight
connect_model.bbox_tower.3.bias
connect_model.bbox_tower.4.weight
connect_model.bbox_tower.4.bias
connect_model.bbox_tower.6.weight
connect_model.bbox_tower.6.bias
connect_model.bbox_tower.7.weight
connect_model.bbox_tower.7.bias
connect_model.bbox_tower.9.weight
connect_model.bbox_tower.9.bias
connect_model.bbox_tower.10.weight
connect_model.bbox_tower.10.bias
connect_model.cls_tower.0.weight
connect_model.cls_tower.0.bias
connect_model.cls_tower.1.weight
connect_model.cls_tower.1.bias
connect_model.cls_tower.3.weight
connect_model.cls_tower.3.bias
connect_model.cls_tower.4.weight
connect_model.cls_tower.4.bias
connect_model.cls_tower.6.weight
connect_model.cls_tower.6.bias
connect_model.cls_tower.7.weight
connect_model.cls_tower.7.bias
connect_model.cls_tower.9.weight
connect_model.cls_tower.9.bias
connect_model.cls_tower.10.weight
connect_model.cls_tower.10.bias
connect_model.cls_memory_tower.0.weight
connect_model.cls_memory_tower.0.bias
connect_model.cls_memory_tower.1.weight
connect_model.cls_memory_tower.1.bias
connect_model.cls_memory_tower.3.weight
connect_model.cls_memory_tower.3.bias
connect_model.cls_memory_tower.4.weight
connect_model.cls_memory_tower.4.bias
connect_model.cls_memory_tower.6.weight
connect_model.cls_memory_tower.6.bias
connect_model.cls_memory_tower.7.weight
connect_model.cls_memory_tower.7.bias
connect_model.cls_memory_tower.9.weight
connect_model.cls_memory_tower.9.bias
connect_model.cls_memory_tower.10.weight
connect_model.cls_memory_tower.10.bias
connect_model.bbox_pred.weight
connect_model.bbox_pred.bias
connect_model.cls_pred.weight
connect_model.cls_pred.bias
connect_model.cls_memory_pred.weight
connect_model.cls_memory_pred.bias
GPU NUM:  1
(WarmUPScheduler) lr spaces: 
[2.50000000e-03 2.87174589e-03 3.29876978e-03 3.78929142e-03
 4.35275282e-03 5.00000000e-03 3.97242620e-03 3.15603398e-03
 2.50742241e-03 1.99211009e-03 1.58270207e-03 1.25743343e-03
 9.99012299e-04 7.93700526e-04 6.30583352e-04 5.00989166e-04
 3.98028497e-04 3.16227766e-04 2.51238292e-04 1.99605115e-04
 1.58583318e-04 1.25992105e-04 1.00098868e-04 7.95270729e-05
 6.31830855e-05 5.01980288e-05 3.98815930e-05 3.16853369e-05
 2.51735325e-05 2.00000000e-05]
model prepare done
train datas: ['GOT10K']
############################################################
<class 'list'>
############################################################
Traceback (most recent call last):
  File "./scripts/train_usot.py", line 367, in <module>
    main()
  File "./scripts/train_usot.py", line 328, in main
    train_set = USOTDataset(config)
  File "/content/drive/MyDrive/USOT/USOT-main/lib/dataset_loader/datasets_usot.py", line 105, in __init__
    self.video_quality, self.far_sample)
  File "/content/drive/MyDrive/USOT/USOT-main/lib/dataset_loader/datasets_usot.py", line 472, in __init__
    self._clean()
  File "/content/drive/MyDrive/USOT/USOT-main/lib/dataset_loader/datasets_usot.py", line 498, in _clean
    frames = self.labels[video]
TypeError: list indices must be integers or slices, not list

Using google collab

zhengjilai commented 2 years ago

@nadavmisgav Hi. Sorry for late reply, as I missed the issue before. Have you run the patch cropping and json generation process successfully as required?