search

AIoT-MLSys-Lab / Efficient-LLMs-Survey

[TMLR 2024] Efficient Large Language Models: A Survey
https://arxiv.org/abs/2312.03863
1.02k stars 85 forks source link

Suggest incorporating one efficient LLM finetuning paper #23

Closed weitianxin closed 10 months ago

weitianxin commented 10 months ago

Thank you for conducting such an insightful survey. I wonder if it's possible to incorporate a recent ICML'23 work from UIUC. It centered on the one-shot compression technique of Pre-trained Langugae Models. This paper investigates the neural tangent kernel (NTK) of the multilayer perceptrons (MLP) modules in a PLM and propose to coin a lightweight PLM through NTK-approximating MLP fusion.

SUSTechBruce commented 10 months ago

Thank you for your valuable advice! Our survey paper and github repository only focuses on billions of parameter LLMs and above, if you have any suggestions for efficient LLMs related papers, please continue to give us new suggestions.