search

nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath
9.11k stars 711 forks source link

Finetuned WizardCoder with LoRA #105

Open shibing624 opened 1 year ago

shibing624 commented 1 year ago

Finetuned WizardCoder with LoRA, repo: https://github.com/shibing624/CodeAssist

Train WizardCoder model

example: examples/training_wizardcoder_mydata.py

cd examples
python training_wizardcoder_mydata.py --do_train --do_predict --num_epochs 1 --output_dir outputs-wizard --model_name WizardLM/WizardCoder-15B-V1.0
sunzx8 commented 1 year ago

May be It did not use lora? I haven't found any words showing that it use lora

shibing624 commented 1 year ago

see the code, use peft to support lora : https://github.com/shibing624/CodeAssist/blob/main/codeassist/wizard_coder.py#L256C17-L256C17