search

InternLM / InternLM-XComposer

InternLM-XComposer2 is a groundbreaking vision-language large model (VLLM) excelling in free-form text-image composition and comprehension.
1.92k stars 121 forks source link

InternLM-XComposer2-4KHD-7B 多卡推理 #274

Open Cloopen-ReLiNK opened 2 months ago

Cloopen-ReLiNK commented 2 months ago

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:0! (when checking argument for argument weight in method wrapper_CUDA__cudnn_convolution)

Kyoani-0718 commented 1 month ago

Did you use lora to fine-tune it? Just load your model like this:

model = AutoPeftModelForCausalLM.from_pretrained(
    '//InternLM-XComposer/finetune/output/lora_finetune_1',
    # device_map="auto",
    trust_remote_code=True
).cuda().eval()

tokenizer = AutoTokenizer.from_pretrained('internlm/internlm-xcomposer2-4khd-7b', trust_remote_code=True)