search

nlpxucan / WizardLM

LLMs build upon Evol Insturct: WizardLM, WizardCoder, WizardMath
9.19k stars 712 forks source link

Prompt template when doing lora fine tune #129

Open gptzerozero opened 1 year ago

gptzerozero commented 1 year ago

Hi, I am learning about QLoRA finetuning and decided to start off by creating a LoRA adapter for the base model WizardLM 30B https://huggingface.co/TheBloke/WizardLM-30B-fp16.

Is there a prompt template available? Does the Alpaca prompt template work for WizardLM 30B?

For example, if I have the question "What is the color of a clear sky? and the answer "A clear sky is blue in color.", how should my dataset row be formatted for QLoRA training?