search

frankaging / Causal-Distill

The Codebase for Causal Distillation for Language Models (NAACL '22)
MIT License
25 stars 3 forks source link

forward() got an unexpected keyword argument 'interchanged_variables' #2

Open Huynh-Chinh opened 2 years ago

Huynh-Chinh commented 2 years ago

Hi @frankaging , when i run causal_training.py i have Error: forward() got an unexpected keyword argument 'interchanged_variables' log 01/14/2022 07:41:12 - INFO - utils - PID: 2342567 - Using MLM loss for LM step. 01/14/2022 07:41:12 - INFO - utils - PID: 2342567 - --- Initializing model optimizer 01/14/2022 07:41:12 - INFO - utils - PID: 2342567 - ------ Number of trainable parameters (student): 91252605 01/14/2022 07:41:12 - INFO - utils - PID: 2342567 - ------ Number of parameters (student): 91450749 01/14/2022 07:41:12 - INFO - utils - PID: 2342567 - Distiller initialization done. 01/14/2022 07:41:12 - INFO - utils - PID: 2342567 - Starting training 01/14/2022 07:41:12 - INFO - utils - PID: 2342567 - --- Starting epoch 0/3 -Iter: 0%| | 0/335884 [00:00<?, ?it/s] Traceback (most recent call last): File "causal_train.py", line 434, in <module> distiller.train() File "/home/chinhh/workspace/Causal-Distill/distillation/causal_distiller.py", line 483, in train is_crossway=self.params.include_crossway, File "/home/chinhh/workspace/Causal-Distill/distillation/causal_distiller.py", line 574, in step skip_update_iter=False, File "/home/chinhh/workspace/Causal-Distill/distillation/causal_distiller.py", line 639, in _step sampled_interchange_position=sampled_interchange_position, File "/home/chinhh/miniconda3/envs/distillation/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) TypeError: forward() got an unexpected keyword argument 'interchanged_variables' please, help me. thanks for releasing the code and paper for your causal distillation approach

frankaging commented 2 years ago

Hi Huynh-Chinh,

Thanks for reaching out!

Sorry, we will be having a major update to the codebase in the next two days. It will definitely fix the issue here. I will post updates again once I merge my dev branch to main.

Best, Zen

frankaging commented 2 years ago

Hi Huynh-Chinh,

We now updated our codebase. Please pull, and follow the updated example script to plug-and-play with your models! Let me know if you have any other questions.

Best, Zen

Huynh-Chinh commented 2 years ago

Hi @frankaging,

Thanks for supporting me!,

I use argument teacher_type roberta and student_type roberta. I have followed your updated. But i still get the same error.

Best, ChinhH

frankaging commented 2 years ago

Hi Huynh-Chinh,

I see. Sorry, it only supports BERT currently. If you want to use Roberta, please follow these two files and create similar model files. I will note this down as a future TODO, i.e., to support other models.

Best, Zen

Huynh-Chinh commented 2 years ago

Hi frankaging,

I hope you will create two models RobertaForMLM and distil-RobertaForMLM soon. Thanks for supporting me!

Best, ChinhH