search

RUCAIBox / LLMSurvey

The official GitHub page for the survey paper "A Survey of Large Language Models".
https://arxiv.org/abs/2303.18223
10.13k stars 798 forks source link

should contain prefix-tunine in chapter 5 #29

Closed applepieiris closed 1 year ago

applepieiris commented 1 year ago

In Chapter 5, besides instruction tuning and alignment tuning, There are also other ways of finetuning LLM to solve different tasks such as prefix-tuning.

I suggest adding it to this review because I think the arrangement now is quite confusing for beginners. More techniques could be covered like blog. The actual trend is "auto prompt, prefix-tuning,P-tuning, prompt tuning(instruction tuning)"

EliverQ commented 1 year ago

Thank you for your suggestion! You can find the added content in the Section 5.3 of our latest version. However, since the theme of our survey is large language models, the aforementioned techniques primarily focus on earlier, smaller pre-trained models, so they may not be extensively covered.

applepieiris commented 1 year ago

Thanks for your reply and efforts. I can find the updated content in the newest version.