search

VHellendoorn / Code-LMs

Guide to using pre-trained large language models of source code
MIT License
1.78k stars 247 forks source link

How to fine tune the models? #25

Closed smith-co closed 2 years ago

smith-co commented 2 years ago

Can we fine tune Polycoder?

smith-co commented 2 years ago

@VHellendoorn can we fine tune Polycoder? Can you provide with the steps?

VHellendoorn commented 2 years ago

Hi, you can fine-tune the model from any of our checkpoints by simply using the GPT-NeoX toolkit with a dataset of your choosing, as long as you point it to the directory containing the checkpoint files and provide the corresponding model parameter config (e.g., 2-7B.yml for the largest model). Please follow the instructions on the aforementioned repository for steps like creating a dataset and training models.