search

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation
https://arxiv.org/abs/2305.07922
BSD 3-Clause "New" or "Revised" License
2.81k stars 419 forks source link

What's the different between Instruction Tuning to Align with Natural Language Instructions and Finetune Using Your Own Data? #130

Open JustinZou1 opened 1 year ago

JustinZou1 commented 1 year ago

What's the different between Instruction Tuning to Align with Natural Language Instructions and Finetune Using Your Own Data?

yuewang-cuhk commented 1 year ago

Hi there, these two are very similar and you can view the instruction tuning as a specific use case of finetuning on your own data. In the instruction tuning example, we showcase how to finetune CodeT5+ 16B on A100-40G with deepspeed and mix precision training, while the finetuning on your own data example is more for smaller models (220M/770M). So if you want to finetune CodeT5+ 2B/6B/16B, the instruction tuning example would be better example to follow.