search

nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath
9.11k stars 711 forks source link

Fine-tune WizCoder on private repos #162

Open bishwenduk029 opened 11 months ago

bishwenduk029 commented 11 months ago

Are there any instructions or documentations on how WizCoder-15B, can be fine-tuned on the private code repos to better help with understanding the code repos and also help with context aware auto-completes. It could have more use cases like automatic PR code reviews etc.

ChiYeungLaw commented 11 months ago

This is an excellent research direction, but we haven't focused much on this topic.

mashdragon commented 11 months ago

Being able to train on the readthedocs and source of the libraries you care most about would be such a helpful improvement and would probably make WizardCoder more effective than even the super large proprietary models.

What steps are required to inject new knowledge into the WizardCoder model? Would fine tuning be enough, or would it be necessary to do a full retrain?