PISSA模式下进行qLORA训练，指定了lora_rank=8，但是训练出来的adaptor_config中是lora_rank=16

[xiningnlp]

Reminder

• [X] I have read the README and searched the existing issues.

System Info

• llamafactory version: 0.8.3.dev0
• Platform: Linux-5.4.0-65-generic-x86_64-with-glibc2.27
• Python version: 3.10.8
• PyTorch version: 2.1.2+cu121 (GPU)
• Transformers version: 4.41.2
• Datasets version: 2.18.0
• Accelerate version: 0.31.0
• PEFT version: 0.11.1
• TRL version: 0.9.4
• GPU type: NVIDIA TITAN RTX
• DeepSpeed version: 0.9.3+195563a2
• vLLM version: 0.4.0

Reproduction

第一步：执行pissa_init.py脚本：

python pissa_init.py pissa_init.py的参数

kwargs = dict(
         model_name_or_path="/data/Qwen/Qwen2-7B-Instruct",
         output_dir="/data/Qwen/Qwen2-7B-Instruct-pissa-r_8",
         lora_rank=8,
         lora_target="k_proj,gate_proj,down_proj,q_proj,v_proj,up_proj,o_proj"
)
quantize_pissa(**kwargs)

生成的目录的pissa_init子目录中的adapter_config.json

{
  "alpha_pattern": {},
  "auto_mapping": null,
  "base_model_name_or_path": "/data/Qwen/Qwen2-7B-Instruct",
  "bias": "none",
  "fan_in_fan_out": false,
  "inference_mode": true,
  "init_lora_weights": true,
  "layer_replication": null,
  "layers_pattern": null,
  "layers_to_transform": null,
  "loftq_config": {},
  "lora_alpha": 16,
  "lora_dropout": 0,
  "megatron_config": null,
  "megatron_core": "megatron.core",
  "modules_to_save": null,
  "peft_type": "LORA",
  "r": 8,
  "rank_pattern": {},
  "revision": null,
  "target_modules": [
    "k_proj",
    "gate_proj",
    "down_proj",
    "q_proj",
    "v_proj",
    "up_proj",
    "o_proj"
  ],
  "task_type": "CAUSAL_LM",
  "use_dora": false,
  "use_rslora": false
}

第二步：进行qLORA训练。

训练重要参数：

pissa_init: false
pissa_iter: 4
pissa_convert: true
bf16: true
quantization_bit: 4
lora_rank: 8
lora_alpha: 16
lora_dropout: 0.05

然而，qLORA训练后，输出目录中的adapter_config.json中的r是错的， r应该是8，而非16

{
  "alpha_pattern": {},
  "auto_mapping": null,
  "base_model_name_or_path": "/data/Qwen/Qwen2-7B-Instruct-pissa-r_8",
  "bias": "none",
  "fan_in_fan_out": false,
  "inference_mode": true,
  "init_lora_weights": true,
  "layer_replication": null,
  "layers_pattern": null,
  "layers_to_transform": null,
  "loftq_config": {},
  "lora_alpha": 32,
  "lora_dropout": 0.05,
  "megatron_config": null,
  "megatron_core": "megatron.core",
  "modules_to_save": null,
  "peft_type": "LORA",
  "r": 16,
  "rank_pattern": {},
  "revision": null,
  "target_modules": [
    "k_proj",
    "up_proj",
    "v_proj",
    "gate_proj",
    "o_proj",
    "down_proj",
    "q_proj"
  ],
  "task_type": "CAUSAL_LM",
  "use_dora": false,
  "use_rslora": false
}

Expected behavior

qLORA训练后，输出目录中的adapter_config.json中的r是错的， r应该是8，而非16

Others

No response

[hiyouga]

我们将 PiSSA 的适配器转换成了普通的 LoRA 适配器，因此 r 就会乘以 2，这是特性： https://github.com/huggingface/peft/tree/main/examples/pissa_finetuning#convert-pissa-to-lora