请教一个问题。如何才能喂饱多个GPU

[leonunix]

提交前必须检查以下项目

• [X] 请确保使用的是仓库最新代码（git pull），一些问题已被解决和修复。
• [X] 我已阅读项目文档和FAQ章节并且已在Issue中对问题进行了搜索，没有找到相似问题和解决方案。
• [X] 第三方插件问题：例如llama.cpp、LangChain、text-generation-webui等，同时建议到对应的项目中查找解决方案。

问题类型

模型训练与精调

基础模型

Chinese-LLaMA-2 (7B/13B)

操作系统

Linux

详细描述问题

你好，开发组，我想请教一个问题。如何才能在单机上把多GPU撑满。情况是这样的，我现在有一台机器上有8块GPU，用你们的脚本跑pt，但是我发现每个gpu的利用率有点低迷，这个需要什么特别的配置吗？ per_device_train_batch_size我设置的是7，超过这个数会报显存不足。

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14              Driver Version: 550.54.14      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 80GB HBM3          Off |   00000000:19:00.0 Off |                    0 |
| N/A   59C    P0            300W /  700W |   80865MiB /  81559MiB |     28%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA H100 80GB HBM3          Off |   00000000:3B:00.0 Off |                    0 |
| N/A   48C    P0            293W /  700W |   80961MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   2  NVIDIA H100 80GB HBM3          Off |   00000000:4C:00.0 Off |                    0 |
| N/A   46C    P0            366W /  700W |   80991MiB /  81559MiB |     29%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   3  NVIDIA H100 80GB HBM3          Off |   00000000:5D:00.0 Off |                    0 |
| N/A   56C    P0            318W /  700W |   80577MiB /  81559MiB |     27%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   4  NVIDIA H100 80GB HBM3          Off |   00000000:9B:00.0 Off |                    0 |
| N/A   58C    P0            366W /  700W |   80561MiB /  81559MiB |     33%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   5  NVIDIA H100 80GB HBM3          Off |   00000000:BB:00.0 Off |                    0 |
| N/A   46C    P0            326W /  700W |   80989MiB /  81559MiB |      1%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   6  NVIDIA H100 80GB HBM3          Off |   00000000:CB:00.0 Off |                    0 |
| N/A   55C    P0            336W /  700W |   80961MiB /  81559MiB |     27%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
|   7  NVIDIA H100 80GB HBM3          Off |   00000000:DB:00.0 Off |                    0 |
| N/A   43C    P0            331W /  700W |   80481MiB /  81559MiB |     32%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A     25052      C   /usr/bin/python3                            80856MiB |
|    1   N/A  N/A     25053      C   /usr/bin/python3                            80952MiB |
|    2   N/A  N/A     25054      C   /usr/bin/python3                            80982MiB |
|    3   N/A  N/A     25055      C   /usr/bin/python3                            80568MiB |
|    4   N/A  N/A     25056      C   /usr/bin/python3                            80552MiB |
|    5   N/A  N/A     25057      C   /usr/bin/python3                            80980MiB |
|    6   N/A  N/A     25058      C   /usr/bin/python3                            80952MiB |
|    7   N/A  N/A     25059      C   /usr/bin/python3                            80472MiB |
+-----------------------------------------------------------------------------------------+

依赖情况（代码类问题务必提供）

# 请在此处粘贴依赖情况（请粘贴在本代码块里）

运行日志或截图

# 请在此处粘贴运行日志（请粘贴在本代码块里）

[iMountTai]

这有多个原因，比如CPU和GPU之间的数据通信瓶颈、模型本身规模小不足以充分利用显卡的能力等，如果您有代码方面的改进和完善，欢迎提出讨论。

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your consideration.

[github-actions[bot]]

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.