TypeError: LoraConfig.init() got an unexpected keyword argument 'enable_lora'

Reminder

[X] I have read the README and searched the existing issues.

System Info

(LargeModelEnv) F:\LargeModel\LLaMA-Factory\LLaMA-Factory\src>llamafactory-cli webui Running on local URL: http://0.0.0.0:7860

To create a public link, set share=True in launch(). 09/04/2024 10:10:37 - WARNING - llamafactory.webui.common - Found complex path, some features may be not available. 09/04/2024 10:15:04 - WARNING - llamafactory.webui.common - Found complex path, some features may be not available. 09/04/2024 10:15:24 - INFO - llamafactory.hparams.parser - Process rank: 0, device: cuda:0, n_gpu: 1, distributed training: False, compute dtype: torch.float16 [INFO|tokenization_utils_base.py:2106] 2024-09-04 10:15:24,607 >> loading file tokenizer.model [INFO|tokenization_utils_base.py:2106] 2024-09-04 10:15:24,607 >> loading file added_tokens.json [INFO|tokenization_utils_base.py:2106] 2024-09-04 10:15:24,607 >> loading file special_tokens_map.json [INFO|tokenization_utils_base.py:2106] 2024-09-04 10:15:24,607 >> loading file tokenizer_config.json [INFO|tokenization_utils_base.py:2106] 2024-09-04 10:15:24,607 >> loading file tokenizer.json 09/04/2024 10:15:25 - INFO - llamafactory.data.template - Add <|user|>,<|observation|> to stop words. 09/04/2024 10:15:25 - INFO - llamafactory.data.template - Cannot add this chat template to tokenizer. 09/04/2024 10:15:25 - INFO - llamafactory.data.loader - Loading dataset identity.json... 09/04/2024 10:15:26 - INFO - llamafactory.data.loader - Loading dataset Requirement_data.json... training example: input_ids: [64790, 64792, 64795, 30910, 13, 14980, 64796, 30910, 13, 13755, 30992, 307, 674, 19622, 30913, 19748, 421, 16158, 297, 21474, 24263, 284, 11265, 9319, 3596, 422, 19622, 1122, 5677, 638, 5890, 2059, 30930, 1072, 457, 307, 2757, 344, 1661, 30987, 2] inputs: [gMASK]sop<|user|> hi<|assistant|> Hello! I am {{a Requirement Inspection Engineer}}, an AI assistant developed by {{beihang university}}. How can I assist you today? label_ids: [2, -100, -100, -100, -100, -100, -100, 30910, 13, 13755, 30992, 307, 674, 19622, 30913, 19748, 421, 16158, 297, 21474, 24263, 284, 11265, 9319, 3596, 422, 19622, 1122, 5677, 638, 5890, 2059, 30930, 1072, 457, 307, 2757, 344, 1661, 30987, 2] labels:

Hello! I am {{a Requirement Inspection Engineer}}, an AI assistant developed by {{beihang university}}. How can I assist you today? [INFO|configuration_utils.py:731] 2024-09-04 10:15:27,265 >> loading configuration file F:\LargeModel\chatglm3-6b-128k\config.json [INFO|configuration_utils.py:731] 2024-09-04 10:15:27,265 >> loading configuration file F:\LargeModel\chatglm3-6b-128k\config.json [INFO|configuration_utils.py:796] 2024-09-04 10:15:27,281 >> Model config ChatGLMConfig { "_name_or_path": "F:\LargeModel\chatglm3-6b-128k", "add_bias_linear": false, "add_qkv_bias": true, "apply_query_key_layer_scaling": true, "apply_residual_connection_post_layernorm": false, "architectures": [ "ChatGLMModel" ], "attention_dropout": 0.0, "attention_softmax_in_fp32": true, "auto_map": { "AutoConfig": "configuration_chatglm.ChatGLMConfig", "AutoModel": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForCausalLM": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSeq2SeqLM": "modeling_chatglm.ChatGLMForConditionalGeneration", "AutoModelForSequenceClassification": "modeling_chatglm.ChatGLMForSequenceClassification" }, "bias_dropout_fusion": true, "classifier_dropout": null, "eos_token_id": 2, "ffn_hidden_size": 13696, "fp32_residual_connection": false, "hidden_dropout": 0.0, "hidden_size": 4096, "kv_channels": 128, "layernorm_epsilon": 1e-05, "model_type": "chatglm", "multi_query_attention": true, "multi_query_group_num": 2, "num_attention_heads": 32, "num_layers": 28, "original_rope": true, "pad_token_id": 0, "padded_vocab_size": 65024, "post_layer_norm": true, "pre_seq_len": null, "prefix_projection": false, "quantization_bit": 0, "rmsnorm": true, "rope_ratio": 500, "seq_length": 131072, "tie_word_embeddings": false, "torch_dtype": "float16", "transformers_version": "4.41.2", "use_cache": true, "vocab_size": 65024 }

[INFO|modeling_utils.py:3471] 2024-09-04 10:15:27,487 >> loading weights file F:\LargeModel\chatglm3-6b-128k\pytorch_model.bin.index.json [INFO|modeling_utils.py:1519] 2024-09-04 10:15:27,487 >> Instantiating ChatGLMForConditionalGeneration model under default dtype torch.float16. [INFO|configuration_utils.py:962] 2024-09-04 10:15:27,487 >> Generate config GenerationConfig { "eos_token_id": 2, "pad_token_id": 0 }

Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [01:38<00:00, 14.09s/it] [INFO|modeling_utils.py:4280] 2024-09-04 10:17:06,279 >> All model checkpoint weights were used when initializing ChatGLMForConditionalGeneration.

[INFO|modeling_utils.py:4288] 2024-09-04 10:17:06,279 >> All the weights of ChatGLMForConditionalGeneration were initialized from the model checkpoint at F:\LargeModel\chatglm3-6b-128k. If your task is similar to the task the model of the checkpoint was trained on, you can already use ChatGLMForConditionalGeneration for predictions without further training. [INFO|configuration_utils.py:915] 2024-09-04 10:17:06,295 >> loading configuration file F:\LargeModel\chatglm3-6b-128k\generation_config.json [INFO|configuration_utils.py:962] 2024-09-04 10:17:06,295 >> Generate config GenerationConfig { "eos_token_id": 2, "pad_token_id": 0 }

09/04/2024 10:17:06 - WARNING - llamafactory.model.model_utils.checkpointing - You are using the old GC format, some features (e.g. BAdam) will be invalid. 09/04/2024 10:17:06 - INFO - llamafactory.model.model_utils.checkpointing - Gradient checkpointing enabled. 09/04/2024 10:17:06 - INFO - llamafactory.model.model_utils.attention - Using vanilla attention implementation. 09/04/2024 10:17:06 - INFO - llamafactory.model.adapter - Upcasting trainable params to float32. 09/04/2024 10:17:06 - INFO - llamafactory.model.adapter - Fine-tuning method: LoRA Traceback (most recent call last): File "C:\SoftWare\Anaconda\AZAnaconda\envs\LargeModelEnv\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\SoftWare\Anaconda\AZAnaconda\envs\LargeModelEnv\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\SoftWare\Anaconda\AZAnaconda\envs\LargeModelEnv\Scripts\llamafactory-cli.exe__main.py", line 7, in File "F:\LargeModel\LLaMA-Factory\LLaMA-Factory\src\llamafactory\cli.py", line 111, in main run_exp() File "F:\LargeModel\LLaMA-Factory\LLaMA-Factory\src\llamafactory\train\tuner.py", line 50, in run_exp run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks) File "F:\LargeModel\LLaMA-Factory\LLaMA-Factory\src\llamafactory\train\sft\workflow.py", line 47, in run_sft model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train) File "F:\LargeModel\LLaMA-Factory\LLaMA-Factory\src\llamafactory\model\loader.py", line 162, in load_model model = init_adapter(config, model, model_args, finetuning_args, is_trainable) File "F:\LargeModel\LLaMA-Factory\LLaMA-Factory\src\llamafactory\model\adapter.py", line 310, in init_adapter model = _setup_lora_tuning( File "F:\LargeModel\LLaMA-Factory\LLaMA-Factory\src\llamafactory\model\adapter.py", line 201, in _setup_lora_tuning model = PeftModel.from_pretrained(model, adapter_to_resume, is_trainable=is_trainable, init_kwargs) File "C:\SoftWare\Anaconda\AZAnaconda\envs\LargeModelEnv\lib\site-packages\peft\peft_model.py", line 371, in from_pretrained config = PEFT_TYPE_TO_CONFIG_MAPPING[ File "C:\SoftWare\Anaconda\AZAnaconda\envs\LargeModelEnv\lib\site-packages\peft\config.py", line 151, in from_pretrained return cls.from_peft_type(kwargs) File "C:\SoftWare\Anaconda\AZAnaconda\envs\LargeModelEnv\lib\site-packages\peft\config.py", line 118, in from_peft_type return config_cls(**kwargs) TypeError: LoraConfig.init__() got an unexpected keyword argument 'enable_lora'

Reproduction

(LargeModelEnv) F:\LargeModel\LLaMA-Factory\LLaMA-Factory\src>llamafactory-cli webui Running on local URL: http://0.0.0.0:7860

Expected behavior

No response

Others

No response

hiyouga / LLaMA-Factory