search

InternLM / InternLM-XComposer

InternLM-XComposer2 is a groundbreaking vision-language large model (VLLM) excelling in free-form text-image composition and comprehension.
1.92k stars 121 forks source link

InternLM-XComposer2-4KHD-7B 多卡推理 #256

Open parasol-ry opened 2 months ago

parasol-ry commented 2 months ago

请问一下 InternLM-XComposer2-4KHD-7B 如何多卡推理，单卡 A100 40G 显存提示不够

LightDXY commented 2 months ago

https://github.com/InternLM/InternLM-XComposer?tab=readme-ov-file#inference-on-multiple-gpus try this one

Cloopen-ReLiNK commented 2 months ago

https://github.com/InternLM/InternLM-XComposer?tab=readme-ov-file#inference-on-multiple-gpus try this one

not work；Why need so much memory? 7B https://github.com/InternLM/InternLM-XComposer/issues/265