A question about fine tuning model type

[Yubo8Zhang]

Hello! When I was fine-tuning the model recently, I noticed that the training script didn't seem to support multi-loss-cocosoda.

I set "model_type=multi-loss-cocosoda" in run_fine_tune.sh. And the script reported the following error:

┌───────────────────── Traceback (most recent call last) ─────────────────────┐ │ E:\github_proj\CoCoSoDa\run.py:1188 in │ │ │ │ 1185 │ │ 1186 │ │ 1187 if name == "main": │ │ > 1188 │ main() │ │ 1189 │ │ 1190 │ │ │ │ E:\github_proj\CoCoSoDa\run.py:1154 in main │ │ │ │ 1151 │ │ │ os.makedirs( output_tokenizer_dir) │ │ 1152 │ │ tokenizer.save_pretrained( output_tokenizer_dir) │ │ 1153 │ if args.do_train: │ │ > 1154 │ │ train(args, model, tokenizer, pool) │ │ 1155 │ │ │ 1156 │ │ │ 1157 │ # Evaluation │ │ │ │ E:\github_proj\CoCoSoDa\run.py:585 in train │ │ │ │ 582 │ │ │ code_inputs = batch[0].to(args.device) │ │ 583 │ │ │ nl_inputs = batch[1].to(args.device) │ │ 584 │ │ │ #get code and nl vectors │ │ > 585 │ │ │ code_vec = model(code_inputs=code_inputs) │ │ 586 │ │ │ nl_vec = model(nl_inputs=nl_inputs) │ │ 587 │ │ │ │ │ 588 │ │ │ tr_num+=1 │ │ │ │ G:\Anaconda3\envs\RetCode\lib\site-packages\torch\nn\modules\module.py:1130 │ │ in _call_impl │ │ │ │ 1127 │ │ # this function, and just call forward. │ │ 1128 │ │ if not (self._backward_hooks or self._forward_hooks or self. │ │ 1129 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hook │ │ > 1130 │ │ │ return forward_call(*input, **kwargs) │ │ 1131 │ │ # Do not call functions when jit is used │ │ 1132 │ │ full_backward_hooks, non_full_backward_hooks = [], [] │ │ 1133 │ │ if self._backward_hooks or _global_backward_hooks: │ └─────────────────────────────────────────────────────────────────────────────┘ TypeError: forward() got an unexpected keyword argument 'code_inputs'

Process finished with exit code 1

[Yubo8Zhang]

I have carefully reviewed the part about fine tuning in your paper. Although it is not clearly stated that the model type used is CoCoSoDa, I think that using CoCoSoDa model type can apply larger batch_size, which usually brings better results in contrastive learning. For people like me who are short of computing resources, it would be very meaningful to support CoCo (Code search with multimodal momentum Contrastive learning) in the fine-tuning phase.

[s1530129650]

Thanks for your question. Please set model_type=base when performing fine-tuning. https://github.com/DeepSoftwareAnalytics/CoCoSoDa/blob/2f2bf8e7994acef846ede7c1078a0b18bc4154d9/run_fine_tune.sh#L7

[Yubo8Zhang]

Okay, I see. So the results in Table II are gained by fine tuning the pertrained model using "model_type=base"?

[s1530129650]

Yeah!😊