model weight file become smaller after prompt tuning?

[Colin97]

Hi, I was prompt tuning "glip_large_model.pth" with the following commands:

python -m torch.distributed.launch --nproc_per_node=1 tools/finetune.py \ --config-file configs/pretrain/glip_Swin_L.yaml \ --ft-tasks configs/odinw_35/10435.yaml \ --skip-test \ --custom_shot_and_epoch_and_general_copy 10_200_4 \ --evaluate_only_best_on_test --push_both_val_and_test \ MODEL.WEIGHT MODEL/glip_large_model.pth \ SOLVER.USE_AMP True \ TEST.DURING_TRAINING True \ SOLVER.IMS_PER_BATCH 3 \ SOLVER.WEIGHT_DECAY 0.25 \ TEST.EVAL_TASK detection \ DATASETS.TRAIN_DATASETNAME_SUFFIX _grounding \ MODEL.BACKBONE.FREEZE_CONV_BODY_AT 2 \ MODEL.DYHEAD.USE_CHECKPOINT True \ SOLVER.TEST_WITH_INFERENCE True \ SOLVER.USE_AUTOSTEP True \ DATASETS.USE_OVERRIDE_CATEGORY True \ SOLVER.SEED 10 \ DATASETS.SHUFFLE_SEED 3 \ DATASETS.USE_CAPTION_PROMPT True \ DATASETS.DISABLE_SHUFFLE True \ SOLVER.STEP_PATIENCE 3 \ SOLVER.CHECKPOINT_PER_EPOCH 1.0 \ SOLVER.AUTO_TERMINATE_PATIENCE 8 \ SOLVER.MODEL_EMA 0.0 \ SOLVER.TUNING_HIGHLEVEL_OVERRIDE full \ SOLVER.BASE_LR 0.05 \ SOLVER.TUNING_HIGHLEVEL_OVERRIDE language_prompt_v2 \ TEST.IMS_PER_BATCH 3 \ SOLVER.FIND_UNUSED_PARAMETERS False

I found that after prompt tuning, the weight file reduced from 6.9GB to 1.7GB. Is that normal? What's the reason behind that?

In order to use the finetuned model, I need to set ADD_LINEAR_LAYER: True. Is there anything else we need to change?

[liunian-harold-li]

Thank you for raising this issue. I will need to taking a closer look. I did not notice this issue before.

For inference, if you are reusing the config from the fine-tuned model, the ADD_LINEAR_LAYER should have been automatically set to true in the tuning_highlevel_override function in finetune.py.

[ccsvd]

i have the same question, when i promt tuning the weight file reduced from 3.5GB to 800M. Is that normal?

[ccsvd]

hi,when i finished the prompt tuning, how to do inference?

[ccsvd]

@Colin97 hi，have you solve the model weight reduced problem？

[KM-github-source]

@Colin97 hi, could u please offer the original "glip_large_model.pth" file to me? I can not get any weight files from https://huggingface.co/GLIPModel/GLIP/blob/main/glip_large_model.pth for unknown reason.

[ccsvd]

这是来自QQ邮箱的假期自动回复邮件。 您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。