The training hyper-parameters for Reproducing Code Generation

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation

https://arxiv.org/abs/2305.07922

BSD 3-Clause "New" or "Revised" License

2.66k stars 391 forks source link

The training hyper-parameters for Reproducing Code Generation #78

Closed xguo7 closed 1 year ago

xguo7 commented 1 year ago

Hello,

I tried to follow the instructions to fine-tune the code generation task in Concode dataset, but failed to get the similar results as shown in the paper. What I got is: Bleu: 26.88, EM: 21.30.

Could you provide all the hyper-parameters regarding the fine-tuning on Concorde datasets? Thank you!

yuewang-cuhk commented 1 year ago

Hi, we have released the training hyper-parameters in the run_exp.py. From your results, the BLEU score is much lower than expected and is problematic. To reproduce the results, please use our released finetuned checkpoints here.