use_peft Not working?

[srn-source]

Full finetuning is working well but use peft is error.

My config is

Model arguments

model_name_or_path: /workspace/sandbox/model_sealion7b

Data training arguments

dataset_mixer:

HuggingFaceH4/ultrafeedback_binarized: 1.0

UCLA-AGI/SPIN_iter0: 1.0 dataset_splits:

• train
• test preprocessing_num_workers: 12

Trainer arguments

bf16: true beta: 0.1 do_eval: false evaluation_strategy: "no" eval_steps: 100 gradient_accumulation_steps: 1 gradient_checkpointing: true

hub_model_id: zephyr-7b-spin

learning_rate: 5.0e-7 log_level: info logging_steps: 10 lr_scheduler_type: linear max_length: 1024 max_prompt_length: 512 num_train_epochs: 3 optim: rmsprop output_dir: outputs per_device_train_batch_size: 1 per_device_eval_batch_size: 1 push_to_hub: false save_strategy: "epoch" save_total_limit: null seed: 42 warmup_ratio: 0.1 use_peft: True lora_r: 16 lora_alpha: 32 lora_dropout: 0.05 lora_target_modules: ["down_proj", "out_proj", "up_proj", "Wqkv"] trust_remote_code: True

I got error message File "/workspace/SPIN/spin/run_spin.py", line 169, in main train_result = spin_trainer.train() File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1555, in train return inner_training_loop( File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1860, in _inner_training_loop tr_loss_step = self.training_step(model, inputs) File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2725, in training_step loss = self.compute_loss(model, inputs) File "/workspace/SPIN/spin/alignment/trainer.py", line 579, in compute_loss loss, metrics = self.get_batch_metrics(model, inputs, train_eval="train") File "/workspace/SPIN/spin/alignment/trainer.py", line 546, in get_batch_metrics ) = self.concatenated_forward(self.ref_model, batch) File "/workspace/SPIN/spin/alignment/trainer.py", line 504, in concatenated_forward all_logits = model( TypeError: 'NoneType' object is not callable

What should I do?

[yihedeng9]

Hi, thank you for your interest! Our codebase has not incorporated lora fine-tuning yet. We will look into this soon.