no run_finetune_rag.sh and missing positional argument: 'logits_processor'

[080152wd]

as it, no run_finetune_rag.sh. i've done all the above in README.md, as that: export HF_HOME and CHECKPOINTS. (many files have been downloaded to cache and ckpt dir and it seems work.) cd scripts ./run_download.sh run_data_preprocessing.sh run_kb_index.sh and up to then. it hasn't received any traceback btw, i didn't create the same conda-env as conda_env.yml do. but have confirmed mainly pkgs the same version. i'll pub that if need as its too long to pub here.

then, as there is no run_finetune_rag.sh, i tried to bash run_finetune_rag_dialdoc.sh. it's maintains ok until a traceback accured after 'validation sanity check'. it seems not a pkg-version error but a coding error. can u give a solution or just a hint for me to modify it?

Validation sanity check: 0%| | 0/2 [00:00<?, ?it/s] Traceback (most recent call last): File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 714, in main(args) File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 676, in main trainer: pl.Trainer = generic_train( File "/datav/my/multidoc2dial/scripts/rag/lightning_base.py", line 389, in generic_train trainer.fit(model) File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 510, in fit results = self.accelerator_backend.train() File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 57, in train return self.train_or_test() File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 74, in train_or_test results = self.trainer.train() File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 532, in train self.run_sanity_check(self.get_model()) File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 731, in run_sanitycheck , eval_results = self.run_evaluation(max_batches=self.num_sanity_val_batches) File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 643, in run_evaluation output = self.evaluation_loop.evaluation_step(batch, batch_idx, dataloader_idx) File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 171, in evaluation_step output = self.trainer.accelerator_backend.validation_step(args) File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 73, in validation_step return self._step(self.trainer.model.validation_step, args) File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 63, in _step output = model_step(*args) File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 321, in validation_step return self._generative_step(batch) File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 363, in _generative_step generated_ids = self.model.generate( File "/datav/my/multidoc2dial/dialdoc/models/rag/modeling_rag_dialdoc.py", line 700, in generate pre_processor = self._get_logits_processor( TypeError: _get_logits_processor() missing 1 required positional argument: 'logits_processor'

[080152wd]

sry, it's my fault. with transformers==4.12.1, it works well. but mine is 4.17. it's pkg-version problem. pls close this issue, thx.

[songfeng]

No worries! Thank you for sharing the issue and the fix!