Closed ngthanhtin closed 2 months ago
hi @ngthanhtin , sorry for the delayed reply.
1) For all LS-*
models, we did not use a prompt.
2) We train the linear head and use LoRA for efficient tuning.
3) For the zero or few shot settings, here is our prompt template:
You are an intelligent text classification system; please help me with the classification task.
Task: identify the classification label from the given sentence.
Only support {{ labels|length }} labels, including: {{ labels|join(', ') }}.
{% if hint is not none %}
Explanation and examples: {{ hint }}
{% endif %}
Output format: label.
Following is the given sentence: {{ text }}
Output:
Thank you so much! @SeanLee97
Hi, Thank you, guys, for providing us with the code, it is very useful for my research.
I have a few questions about your training:![Screenshot 2024-06-21 at 1 36 20 PM](https://github.com/4AI/LS-LLaMA/assets/17596799/5f820c33-8aee-4792-873a-a69739eb96c2)
1/ Here, what is the prompt for LS-LLaMA-2-7B? and where can I find this? 2/ Do you train only the linear classification head or fine-tune the whole Llama model? 3/Could you provide us with the zero and few-shot Llama code?
I hope you can answer my question, it is fascinating to find out these.
Best, Tin