search

airockchip / rknn-llm

Other
347 stars 29 forks source link

support qwen2-7b? #70

Open iamananba opened 3 months ago

iamananba commented 3 months ago

support qwen2-7b?

LCW0NJUPT commented 2 months ago

same question

BUG1989 commented 1 month ago

这个档位的芯片，就不要指望跑 7B 的模型了吧，即使跑起来了，也就 2-3个token/s，有啥意义呢？

BUG1989 commented 1 month ago

要不试试隔壁 AXera 的端侧大模型部署？至少不会“胡说八道” https://github.com/AXERA-TECH/ax-llm