[Feature] LMDeploy部署 tp只能支持2的n次方吗

OpenGVLab / InternVL

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

https://internvl.readthedocs.io/en/latest/

MIT License

5.63k stars 439 forks source link

Closed luo7297605 closed 3 weeks ago

luo7297605 commented 3 weeks ago

LMDeploy部署 tp只能支持2的n次方吗，因为现在5张卡想要部署internvl 26b

No response

No response

irexyc commented 3 weeks ago

kv_head的数量是8，没办法被5整除。