OD weight for pretraining missmatch

JacobYuan7 / RLIPv2

[ICCV 2023] RLIPv2: Fast Scaling of Relational Language-Image Pre-training

Apache License 2.0

112 stars 3 forks source link

OD weight for pretraining missmatch #12

Closed safsfsvvea closed 6 months ago

safsfsvvea commented 7 months ago

when I use the shell train_RLIP_ParSeDA_v2_mixed_vgcocoo365_swinL.sh to reproduce the pretraining process, I set the --pretrained to the OD weight I downloaded swin_large_cocoo365_bs64_lr141_drop_path0.5_dp0_mqs_lft_dab_deformable_detr_plus_iterative_bbox_refinement_36eps_converted.pth. However, there are many missing_keys. I wonder which checkpoint is the correct one for this shell.

JacobYuan7 commented 7 months ago

@safsfsvvea Hi, actually there should be some mismatches since there are extra components introduced on top of detection parameters. If you use RLIP parameters for HOI fine-tuning, then there should not be any mismatch.