search

sail-sg / closer-look-LLM-unlearning

The official code of the paper "A Closer Look at Machine Unlearning for Large Language Models".
https://arxiv.org/abs/2410.08109
14 stars 1 forks source link

TypeError: CustomTrainerForgetting.compute_loss() got an unexpected keyword argument 'num_items_in_batch' #1

Open liuyugeng opened 2 days ago

liuyugeng commented 2 days ago

Hi, when I try to train the model, it returns the following error. 

Traceback (most recent call last):
  File "closer-look-LLM-unlearning/forget.py", line 245, in main
    trainer.train()
  File "anaconda3/envs/tofu/lib/python3.10/site-packages/transformers/trainer.py", line 2123, in train
    return inner_training_loop(
  File "anaconda3/envs/tofu/lib/python3.10/site-packages/transformers/trainer.py", line 2481, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
  File "anaconda3/envs/tofu/lib/python3.10/site-packages/transformers/trainer.py", line 3579, in training_step
    loss = self.compute_loss(model, inputs, num_items_in_batch=num_items_in_batch)
TypeError: CustomTrainerForgetting.compute_loss() got an unexpected keyword argument 'num_items_in_batch'

I just followed the instructions bash scripts/continual_tofu/idk_ap.sh.

liuyugeng commented 2 days ago

I think in the compute_loss function of trainer class, you should add num_items_in_batch=None.

liuyugeng commented 14 hours ago

I just upgraded to the latest library (4.46.3) and found this issue.

LetheSec commented 13 hours ago

Oh, you're right. The version of transformers should be <4.46, and we used 4.43.0 in our experiments.