search

WisdomShell / codeshell

A series of code large language models developed by PKU-KCL
http://se.pku.edu.cn/kcl
Other
1.61k stars 120 forks source link

run_finetune.sh 微调训练 需要多少张卡 #41

Open wenma1993 opened 11 months ago

wenma1993 commented 11 months ago

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.76 GiB. GPU 4 has a total capacty of 23.65 GiB of which 1.36 GiB is free. Process 2157808 has 22.28 GiB memory in use. Of the allocated memory 16.34 GiB is allocated by PyTorch, and 5.38 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

用了4090 24G 4张 都报不足 ，如何解决更好呢 谢谢

wenma1993 commented 11 months ago

基础模型是 ：CodeShell-7B-Chat

ruixie commented 11 months ago

你好，7B模型训练时，需要大概100G以上的显存。

wenma1993 commented 11 months ago

可以弄一个讨论群吗 ？

whyPeanutbutter commented 11 months ago

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.76 GiB. GPU 4 has a total capacty of 23.65 GiB of which 1.36 GiB is free. Process 2157808 has 22.28 GiB memory in use. Of the allocated memory 16.34 GiB is allocated by PyTorch, and 5.38 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

用了4090 24G 4张 都报不足 ，如何解决更好呢 谢谢

麻烦问下，进行推理的时候如何指定在多张卡上运行？🙏我的只运行在单张卡上，万分感谢🙏

yiyepiaoling0715 commented 10 months ago

你好，7B模型训练时，需要大概100G以上的显存。

现在有群了么

yiyepiaoling0715 commented 10 months ago

你好，7B模型训练时，需要大概100G以上的显存。

现在有群了么

yiyepiaoling0715，方便加一下?

philipgao518 commented 10 months ago

8卡24GB，微调爆显存，有知道为什么的吗？

philipgao518 commented 10 months ago

8卡24GB，微调爆显存，有知道为什么的吗？

找到原因了，finetune没有默认开启lora，开启lora以后8卡每卡8GB显存够了

xianqiangHub commented 3 months ago

8卡24GB，微调爆显存，有知道为什么的吗？

找到原因了，finetune没有默认开启lora，开启lora以后8卡每卡8GB显存够了

是您自己写的lora和原模型合并的代码吗