search

HarderThenHarder / transformers_tasks

⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.
https://www.zhihu.com/column/c_1451236880973426688
2.11k stars 376 forks source link

基于lora的方式finetune模型后，推理时间很长 #80

Open wangjiangyue0226 opened 1 year ago

wangjiangyue0226 commented 1 year ago

基于lora的方式finetune模型后，推理时间很长，平均时间要40s，而相同的case直接用chatglm-6b推理平均时间要16s