非量化版本的codegeex2-6B，16GB显存下最大推理的max_new_tokens约为多少？微调后经常推着推着就OOM了

THUDM / CodeGeeX2

CodeGeeX2: A More Powerful Multilingual Code Generation Model

https://codegeex.cn

Apache License 2.0

7.55k stars 532 forks source link

Open CatYing opened 3 months ago

CatYing commented 3 months ago

as topic temperature=0.2, max_new_tokens=4097

PineREN commented 3 months ago

求问怎么微调的，是用的chatglmv2的代码进行微调的吗？