search

EvolvingLMMs-Lab / lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval
https://lmms-lab.github.io/
Other
1.43k stars 123 forks source link

Try on qwen2vl-72b, out of memory #285

Open ChinChyi opened 2 weeks ago

ChinChyi commented 2 weeks ago

when I use 8*A100 to test Qwen2VL-72b model, I use accelerate launch --num_processes=8 -m lmms_eval --model qwen2_vl --model_args pretrained="Qwen/Qwen2-VL-72B-Instruct" --tasks xxx --batch_size 1 --log_samples --log_samples_suffix qwen2_vl_72b --output_path ./logs/ but I meet this issue: image

kcz358 commented 1 week ago

Can you try this?

accelerate launch --num_processes=1 -m lmms_eval --model qwen2_vl  --model_args pretrained="Qwen/Qwen2-VL-72B-Instruct",device_map=auto   --tasks xxx --batch_size 1 --log_samples --log_samples_suffix qwen2_vl_72b --output_path ./logs/
NimrodShabtay commented 4 days ago

I have the same issue, with other models as well (InternVL2-76B), the device_map argument didn't help for me