search

AGI-Edgerunners / LLM-Adapters

Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"
https://arxiv.org/abs/2304.01933
Apache License 2.0
1.05k stars 99 forks source link

AttributeError: 'tuple' object has no attribute 'update' #60

Open YananLi18 opened 6 months ago

YananLi18 commented 6 months ago

I have uninstalled the official peft package and used the version you provided. Unfortunately, I encountered the following problem:

Traceback (most recent call last): File "finetune.py", line 350, in fire.Fire(train) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/fire/core.py", line 143, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/fire/core.py", line 477, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace component = fn(varargs, kwargs) File "finetune.py", line 317, in train trainer.train(resume_from_checkpoint=resume_from_checkpoint) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/trainer.py", line 1624, in train return inner_training_loop( File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/trainer.py", line 1961, in _inner_training_loop tr_loss_step = self.training_step(model, inputs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/trainer.py", line 2902, in training_step loss = self.compute_loss(model, inputs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/trainer.py", line 2925, in compute_loss outputs = model(inputs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(input, kwargs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/accelerate/utils/operations.py", line 822, in forward return model_forward(*args, *kwargs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/accelerate/utils/operations.py", line 810, in call return convert_to_fp32(self.model_forward(args, kwargs)) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast return func(args, kwargs) File "/data/yananli/lora_transfer/LLM-Adapters/peft/src/peft/peft_model.py", line 568, in forward return self.base_model(input_ids=input_ids, past_key_values=past_key_values, kwargs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(input, kwargs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1176, in forward outputs = self.model( File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, *kwargs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1019, in forward layer_outputs = decoder_layer( File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(input, kwargs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 740, in forward hidden_states, self_attn_weights, present_key_value = self.self_attn( File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, **kwargs) File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 367, in forward key_states, value_states = past_key_value.update(key_states, value_states, self.layer_idx, cache_kwargs) AttributeError: 'tuple' object has no attribute 'update'

It seems that the past_key_value returned by self.get_prompt(batch_size) of PeftModelForCausalLM is a tuple but a dictionary is needed. Can you give me any advice on how to fix this on your specific peft? or The torch and transformers versions you recommended?

My current environment versions are: pytorch 1.13.1
pytorch-cuda 11.7
pytorch-mutex 1.0
torch 2.2.1
torchaudio 0.13.1
torchvision 0.14.1
transformers 4.38.2

Thank you for your attention and looking forward to your reply

HZQ950419 commented 6 months ago

Hi, Can I ask what kind of adapter are you using and the command used to run finetune.py?

YananLi18 commented 6 months ago

Hi, Can I ask what kind of adapter are you using and the command used to run finetune.py?

I want to use prefix tuning and the command used to run finetune.py is: CUDA_VISIBLE_DEVICES=4 python finetune.py --base_model '../llama-7b-hf/' --data_path './ft-training_set/math_10k.json' --output_dir './trained_models/llama-7B-math-prefix' --batch_size 16 --micro_batch_size 4 --num_epochs 3 --learning_rate 3e-4 --cutoff_len 256 --val_set_size 120 --adapter_name prefix-tuning --eval_step 10 --save_step 10 --num_virtual_tokens 10

HZQ950419 commented 6 months ago

For prefix-tuning, I can run the command with transformers==4.35. The later versions will cause different errors, for example https://github.com/huggingface/peft/pull/1252. Please try with transformers==4.35.

YananLi18 commented 6 months ago

For prefix-tuning, I can run the command with transformers==4.35. The later versions will cause different errors, for example huggingface/peft#1252. Please try with transformers==4.35.

Thanks for your quick reply. It works well with transformers==4.35 installed.

By the way, the specific peft module you provided is built on top with which official version?

HZQ950419 commented 6 months ago

The LLM-Adapters framework is built on top of peft==0.3.0.