[!NOTE]
Please avoid creating issues regarding the following questions, as they might be closed without a response.
请避免创建与下述问题有关的 issues，这些 issues 可能不会被回复。
The out-of-memory (OOM) error during training is usually due to insufficient space of the current device to complete the computation. You can try the following methods to deal with this issue:
Reduce the training batch size per_device_train_batch_size: 1
Reduce the maximum sequence length cutoff_len: 512
Set quantization_bit: 4 to quantize model parameters (only compatible with LoRA tuning)
Use DeepSpeed ZeRO-3 or FSDP to partition model weights on multiple devices
Use the paged optimizer optim: paged_adamw_8bit
模型训练时显存溢出，通常是由于当前某个设备的剩余显存不足以完成计算任务。可尝试下述方法解决：
降低批处理大小 per_device_train_batch_size: 1
降低最大序列长度 cutoff_len: 256
设置 quantization_bit: 4 量化模型参数（仅限于 LoRA 方法）
使用 DeepSpeed ZeRO-3 或 FSDP 将模型权重拆分到多个设备
使用分页低精度优化器 optim: paged_adamw_8bit
Unsatisfying fine-tuning results / 微调效果无法令人满意
Unsatisfying fine-tuning results are usually due to insufficient training samples, leading to underfitting. You can try the following methods to deal with this issue:
Increase the size of the training dataset
Increase the number of epochs num_train_epochs: 5.0 or steps max_steps: 1000
Use a larger learning rate learning_rate: 2.0e-4
微调效果较差，通常是由于训练样本过少，导致模型欠拟合。可尝试下述方法解决：
提高训练数据集的大小
提高训练轮数 num_train_epochs: 5.0 或步数 max_steps: 1000
增大学习率 learning_rate: 2.0e-4
Corrupted or repeated model responses / 胡乱或重复的模型回答
If this issue occurs before training, it is usually due to using an unaligned (base) model or a mismatched template. Please ensure an aligned (instruct/chat) model and correct template are used.
If this issue occurs after training, please check if the template used for training and inference is consistent. And do not forget to check if the overfitting appeared.
The error when using bf16 is due to some devices (e.g., NPUs) does not support bfloat16. Please replace bf16 with fp16.
If the error occurs on GPU devices, please use the following command to check if the CUDA version of PyTorch is installed correctly:
Please ensure that the working directory when launching the LLaMA Board is the same as the LLaMA-Factory directory.
请确保启动 LLaMA Board 时的工作目录与 LLaMA-Factory 主目录一致。
How to shard model weights on multiple devices / 如何模型权重拆分到多个设备上
During the training phase, please refer to the examples about how to use the DeepSpeed ZeRO-3 (recommended) or FSDP.
During the inference phase, the model will be automatically loaded onto all available devices.
Out-of-memory / 显存溢出
The out-of-memory (OOM) error during training is usually due to insufficient space of the current device to complete the computation. You can try the following methods to deal with this issue:
per_device_train_batch_size: 1cutoff_len: 512quantization_bit: 4to quantize model parameters (only compatible with LoRA tuning)optim: paged_adamw_8bit模型训练时显存溢出，通常是由于当前某个设备的剩余显存不足以完成计算任务。可尝试下述方法解决：
per_device_train_batch_size: 1cutoff_len: 256quantization_bit: 4量化模型参数（仅限于 LoRA 方法）optim: paged_adamw_8bitUnsatisfying fine-tuning results / 微调效果无法令人满意
Unsatisfying fine-tuning results are usually due to insufficient training samples, leading to underfitting. You can try the following methods to deal with this issue:
num_train_epochs: 5.0or stepsmax_steps: 1000learning_rate: 2.0e-4微调效果较差，通常是由于训练样本过少，导致模型欠拟合。可尝试下述方法解决：
num_train_epochs: 5.0或步数max_steps: 1000learning_rate: 2.0e-4Corrupted or repeated model responses / 胡乱或重复的模型回答
If this issue occurs before training, it is usually due to using an unaligned (base) model or a mismatched
template. Please ensure an aligned (instruct/chat) model and correcttemplateare used. If this issue occurs after training, please check if thetemplateused for training and inference is consistent. And do not forget to check if the overfitting appeared.若该问题发生在训练之前，通常是由于使用了未经对齐（base）的模型或不恰当的模板
template，请保证使用对齐后（instruct/chat）的模型和正确的模板template。 若该问题发生在训练之后，请检查训练和推理使用的模板template是否一致，同时检查是否发生了过拟合。llamafactory-cli: command not found / 无法找到命令
Please install LLaMA Factory using pip according to the README. If errors persist after installation, try restarting the terminal.
请根据 README 使用 pip 安装 LLaMA Factory。若安装后仍出现报错，请尝试重启终端。
Mixed precision training problems / 混合精度训练报错
The error when using
bf16is due to some devices (e.g., NPUs) does not support bfloat16. Please replacebf16withfp16. If the error occurs on GPU devices, please use the following command to check if the CUDA version of PyTorch is installed correctly:Mixed precision training is not supported on the CPU and Mac. Please remove the
bf16orfp16parameter and setlow_cpu_mem_usage: false.使用
bf16时出现错误，是由于部分计算设备（例如 NPU）不支持 bfloat16，请将bf16更换为fp16。 若在 GPU 设备上出现半精度错误，请使用下述命令检查 CUDA 版本的 PyTorch 是否被正确安装：CPU 和 Mac 芯片尚未支持混合精度训练，请移除
bf16或fp16参数并设置low_cpu_mem_usage: false。LLaMA Board cannot display datasets / LLaMA Board 无法显示数据集
Please ensure that the working directory when launching the LLaMA Board is the same as the LLaMA-Factory directory.
请确保启动 LLaMA Board 时的工作目录与 LLaMA-Factory 主目录一致。
How to shard model weights on multiple devices / 如何模型权重拆分到多个设备上
During the training phase, please refer to the examples about how to use the DeepSpeed ZeRO-3 (recommended) or FSDP. During the inference phase, the model will be automatically loaded onto all available devices.
在训练阶段，请参考 examples 使用 DeepSpeed ZeRO-3（推荐）或 FSDP。 在推理阶段，模型会被自动加载到所有可用的设备上。
Distributed training stuck / 分布式训练卡住
Try setting the environment variable
export NCCL_P2P_LEVEL=NVL.请尝试设置环境变量
export NCCL_P2P_LEVEL=NVL。How to use ORPO or SimPO / 如何使用 ORPO 或 SimPO
Modify the
pref_lossin example script toorpoorsimpo.将示例脚本 中的
pref_loss改为orpo或simpo。