search

airockchip / rknn-llm

Other
258 stars 26 forks source link

模型转换内存需求过大 #30

Open YanxingLiu opened 2 months ago

YanxingLiu commented 2 months ago

从 hf 模型转换到 rkllm 模型时候所需要的内存太大了，后续能否优化下，目前 32g 的机器只够转换 1.5B 的模型

fydeos-alex commented 2 months ago

I found that we can download some models from Huggingface now. Just search "rkllm"

YanxingLiu commented 2 months ago

I found that we can download some models from Huggingface now. Just search "rkllm"

ok, thanks for your notification, I have found some other models

waydong commented 2 months ago

正式版会解决这个问题

Tidus1991 commented 2 months ago

实测7b推理的速度只有2token/s出头，很难用

zhoujing07 commented 2 months ago

加大swapfile即可

AACengineer commented 2 months ago

加大swapfile即可

7B模型，32G内存，swapfile应该设置多少？