search

alibaba / Pai-Megatron-Patch

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.
Apache License 2.0
674 stars 94 forks source link

模型转换显存占用问题 #287

Closed coder-wangzhen closed 2 months ago

coder-wangzhen commented 2 months ago

感谢开源，我在使用hf2mcore_qwen2_convertor.sh转7B模型时在24G显存的A10上报OOM，我尝试转更小的1.5B模型发现显存占用大约为9G，请问可以使用多卡转换或卸载到cpu内存转换吗？72B模型单卡A100/A800 80G显存能转换的吗？

LJLQ commented 2 months ago

hf2mcore_qwen2_convertor.sh 里面改 cpu_options=" \ --use-cpu-initialization" 好像可以使用cpu

coder-wangzhen commented 2 months ago

hf2mcore_qwen2_convertor.sh 里面改 cpu_options=" --use-cpu-initialization" 好像可以使用cpu

@LJLQ 嗯是的，多谢