linalg.svd: The input tensor A must have at least 2 dimensions.

[q497629642]

你好，我加载Qwen1.5-72B-Chat模型，报

model = get_peft_model(model, lora_config) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/mapping.py", line 149, in get_peft_model model = get_peft_model(model, lora_config)model = get_peft_model(model, lora_config)

File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/mapping.py", line 149, in get_peft_model File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/mapping.py", line 149, in get_peft_model return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/peft_model.py", line 1395, in init return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name)return MODEL_TYPE_TOPEFT MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name)

File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/peft_model.py", line 1395, in init File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/peft_model.py", line 1395, in init super().init(model, peft_config, adapter_name) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/peft_model.py", line 138, in init super().init(model, peft_config, adapter_name)super().init__(model, peft_config, adapter_name)

File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/peft_model.py", line 138, in init File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/peft_model.py", line 138, in init self.base_model = cls(model, {adapter_name: peft_config}, adapter_name) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 139, in in it self.base_model = cls(model, {adapter_name: peft_config}, adapter_name) self.base_model = cls(model, {adapter_name: peft_config}, adapter_name) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0 .10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 139, in init

File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 139, in init super().init(model, config, adapter_name) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/tuners_utils.py", line 166, in init super().init(model, config, adapter_name) super().init(model, config, adapter_name) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft /tuners/tuners_utils.py", line 166, in init

File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/tuners_utils.py", line 166, in init self.inject_adapter(self.model, adapter_name) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/tuners_utils.py", line 372, in inject _adapter self.inject_adapter(self.model, adapter_name) self.inject_adapter(self.model, adapter_name) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft /tuners/tuners_utils.py", line 372, in inject_adapter

File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/tuners_utils.py", line 372, in inject _adapter self._create_and_replace(peft_config, adapter_name, target, target_name, parent, current_key=key) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 223, in _cre ate_and_replace self._create_and_replace(peft_config, adapter_name, target, target_name, parent, current_key=key) self._create_and_replace(peft_config, adapter_name, target, target_name, parent, current_key=key) File "/data1/anaconda/env/ysj_qwen/lib/pytho n3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 223, in _create_and_replace

File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 223, in create and_replace new_module = self._create_new_module(lora_config, adapter_name, target, kwargs) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 314, in create new_module new_module = self._create_new_module(lora_config, adapter_name, target, kwargs) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 314, in _cre ate_new_module new_module = self._create_new_module(lora_config, adapter_name, target, kwargs) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/model.py", line 314, in create new_module new_module = dispatcher(target, adapter_name, lora_config=lora_config, kwargs) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 1116, in dispatc h_default new_module = dispatcher(target, adapter_name, lora_config=lora_config, kwargs) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 1116, in dispatc h_default new_module = Linear(target, adapter_name, kwargs) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 410, in init new_module = Linear(target, adapter_name, kwargs) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 410, in init new_module = Linear(target, adapter_name, kwargs) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 410, in init self.update_layer( File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 117, in update_l ayer self.update_layer( File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 117, in update_l ayer self.update_layer( File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 117, in update_l ayer self.pissa_init(adapter_name, init_lora_weights) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 173, in pissa_in it self.pissa_init(adapter_name, init_lora_weights) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 173, in pissa_in it self.pissa_init(adapter_name, init_lora_weights) File "/data1/anaconda/env/ysj_qwen/lib/python3.10/site-packages/peft-0.10.1.dev0-py3.10.egg/peft/tuners/lora/layer.py", line 173, in pissa_in it V, S, Uh = torch.linalg.svd(weight.data, full_matrices=False) RuntimeError : V, S, Uh = torch.linalg.svd(weight.data, full_matrices=False)linalg.svd: The input tensor A must have at least 2 dimensions.

RuntimeErrorV, S, Uh = torch.linalg.svd(weight.data, full_matrices=False):

linalg.svd: The input tensor A must have at least 2 dimensions.

[fxmeng]

您好， 我在本地试着初始化Qwen1.5-72B-Chat，并未出现您的这个问题。建议您：

1. PiSSA已经本merge进了peft主分支，您可以升级到最新版本的PiSSA尝试问题是否被解决：pip install -U peft
2. 检查您Qwen1.5-72B-Chat的下载是否完整
3. 在报错处插入断点，如打印weight.data的形状等信息
4. 我们将会初始化并上传PiSSA-Qwen1.5-72B-Chat到huggingface，您可以提供你所需的配置，我将优先处理这个需求。

[SYfengh]

Qwen1.5-7B-chat和14B-chat也有同样的问题，peft最新版本（deepspeed zero3加载模型时会出现此问题，zero2正常；zero3时打印model.param全为空）

[fxmeng]

linalg.svd: The input tensor A must have at least 2 dimensions. 当target_modules中bias=True时，会出现这个错误，解决方法是设置target_modules=["_proj.weight"]，这样会过滤掉dimensions为1的bias。 关于deepspeed zero3加载模型的问题，请问是任意模型都会出现，还是仅仅这两个模型才会出现？