How to train SPM in stage2?

[Lich-King000]

Thank you for your excellent work. I have some questions about the training process of SPM.

I encounter a problem when I use the script in train_mixformer.sh to train SPM module. python tracking/train.py --script mixformer_online --config baseline --save_dir /mysavepath --mode multiple --nproc_per_node 1 --stage1_model /mylatest checkpoint trained in the first stage

But the logs show that it seems that the program has loaded wrong checkpoint because there are so many missing keys

missing keys: ['score_branch.score_token', 'score_branch.score_head.layers.0.weight', 'score_branch.score_head.layers.0.bias', 'score_branch.score_head.layers.1.weight', 'score_branch.score_head.layers.1.bias', 'score_branch.score_head.layers.2.weight', 'score_branch.score_head.layers.2.bias', 'score_branch.proj_q.0.weight', 'score_branch.proj_q.0.bias', 'score_branch.proj_q.1.weight', 'score_branch.proj_q.1.bias', 'score_branch.proj_k.0.weight', 'score_branch.proj_k.0.bias', 'score_branch.proj_k.1.weight', 'score_branch.proj_k.1.bias', 'score_branch.proj_v.0.weight', 'score_branch.proj_v.0.bias', 'score_branch.proj_v.1.weight', 'score_branch.proj_v.1.bias', 'score_branch.proj.0.weight', 'score_branch.proj.0.bias', 'score_branch.proj.1.weight', 'score_branch.proj.1.bias', 'score_branch.norm1.weight', 'score_branch.norm1.bias', 'score_branch.norm2.0.weight', 'score_branch.norm2.0.bias', 'score_branch.norm2.1.weight', 'score_branch.norm2.1.bias'] unexpected keys: [] Loading pretrained mixformer weights done.

I am really confused about how to train the SPM module correctly.

I am appreciate if you can give me some advices.

The whole log shows below:

error logs.txt

[laisimiao]

It's right, the missing keys are exactly those params that need to be trained

[yutaocui]

Thank you for your excellent work. I have some questions about the training process of SPM.

I encounter a problem when I use the script in train_mixformer.sh to train SPM module. python tracking/train.py --script mixformer_online --config baseline --save_dir /mysavepath --mode multiple --nproc_per_node 1 --stage1_model /mylatest checkpoint trained in the first stage

But the logs show that it seems that the program has loaded wrong checkpoint because there are so many missing keys

missing keys: ['score_branch.score_token', 'score_branch.score_head.layers.0.weight', 'score_branch.score_head.layers.0.bias', 'score_branch.score_head.layers.1.weight', 'score_branch.score_head.layers.1.bias', 'score_branch.score_head.layers.2.weight', 'score_branch.score_head.layers.2.bias', 'score_branch.proj_q.0.weight', 'score_branch.proj_q.0.bias', 'score_branch.proj_q.1.weight', 'score_branch.proj_q.1.bias', 'score_branch.proj_k.0.weight', 'score_branch.proj_k.0.bias', 'score_branch.proj_k.1.weight', 'score_branch.proj_k.1.bias', 'score_branch.proj_v.0.weight', 'score_branch.proj_v.0.bias', 'score_branch.proj_v.1.weight', 'score_branch.proj_v.1.bias', 'score_branch.proj.0.weight', 'score_branch.proj.0.bias', 'score_branch.proj.1.weight', 'score_branch.proj.1.bias', 'score_branch.norm1.weight', 'score_branch.norm1.bias', 'score_branch.norm2.0.weight', 'score_branch.norm2.0.bias', 'score_branch.norm2.1.weight', 'score_branch.norm2.1.bias'] unexpected keys: [] Loading pretrained mixformer weights done.

I am really confused about how to train the SPM module correctly.

I am appreciate if you can give me some advices.

The whole log shows below:

error logs.txt

The error comes from your incorrect training datasets config 'TRAIN': {'DATASETS_NAME': ['TNL2K_train'], 'DATASETS_RATIO': [1, 1, 1, 1]. It should be 'TRAIN': {'DATASETS_NAME': ['TNL2K_train'], 'DATASETS_RATIO': [1]

[Lich-King000]

Thank you for your advices, it solved.

[samueleruffino99]

Hello, I would like to fine-tuning at stage 2 MicFormer ConvMAE Online Large, is there pretrained model (after stage 1 training)?