as it, no run_finetune_rag.sh.
i've done all the above in README.md, as that:
export HF_HOME and CHECKPOINTS. (many files have been downloaded to cache and ckpt dir and it seems work.)
cd scripts
./run_download.sh
run_data_preprocessing.shrun_kb_index.sh
and up to then. it hasn't received any traceback
btw, i didn't create the same conda-env as conda_env.yml do. but have confirmed mainly pkgs the same version. i'll pub that if need as its too long to pub here.
then, as there is no run_finetune_rag.sh, i tried to bash run_finetune_rag_dialdoc.sh. it's maintains ok until a traceback accured after 'validation sanity check'. it seems not a pkg-version error but a coding error.
can u give a solution or just a hint for me to modify it?
Validation sanity check: 0%| | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 714, in
main(args)
File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 676, in main
trainer: pl.Trainer = generic_train(
File "/datav/my/multidoc2dial/scripts/rag/lightning_base.py", line 389, in generic_train
trainer.fit(model)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 510, in fit
results = self.accelerator_backend.train()
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 57, in train
return self.train_or_test()
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 74, in train_or_test
results = self.trainer.train()
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 532, in train
self.run_sanity_check(self.get_model())
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 731, in run_sanitycheck
, eval_results = self.run_evaluation(max_batches=self.num_sanity_val_batches)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 643, in run_evaluation
output = self.evaluation_loop.evaluation_step(batch, batch_idx, dataloader_idx)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 171, in evaluation_step
output = self.trainer.accelerator_backend.validation_step(args)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 73, in validation_step
return self._step(self.trainer.model.validation_step, args)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 63, in _step
output = model_step(*args)
File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 321, in validation_step
return self._generative_step(batch)
File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 363, in _generative_step
generated_ids = self.model.generate(
File "/datav/my/multidoc2dial/dialdoc/models/rag/modeling_rag_dialdoc.py", line 700, in generate
pre_processor = self._get_logits_processor(
TypeError: _get_logits_processor() missing 1 required positional argument: 'logits_processor'
as it, no run_finetune_rag.sh. i've done all the above in README.md, as that: export HF_HOME and CHECKPOINTS. (many files have been downloaded to cache and ckpt dir and it seems work.) cd scripts ./run_download.sh run_data_preprocessing.sh run_kb_index.sh and up to then. it hasn't received any traceback btw, i didn't create the same conda-env as conda_env.yml do. but have confirmed mainly pkgs the same version. i'll pub that if need as its too long to pub here.
then, as there is no run_finetune_rag.sh, i tried to bash run_finetune_rag_dialdoc.sh. it's maintains ok until a traceback accured after 'validation sanity check'. it seems not a pkg-version error but a coding error. can u give a solution or just a hint for me to modify it?
Validation sanity check: 0%| | 0/2 [00:00<?, ?it/s] Traceback (most recent call last): File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 714, in
main(args)
File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 676, in main
trainer: pl.Trainer = generic_train(
File "/datav/my/multidoc2dial/scripts/rag/lightning_base.py", line 389, in generic_train
trainer.fit(model)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 510, in fit
results = self.accelerator_backend.train()
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 57, in train
return self.train_or_test()
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/accelerator.py", line 74, in train_or_test
results = self.trainer.train()
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 532, in train
self.run_sanity_check(self.get_model())
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 731, in run_sanitycheck
, eval_results = self.run_evaluation(max_batches=self.num_sanity_val_batches)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/trainer.py", line 643, in run_evaluation
output = self.evaluation_loop.evaluation_step(batch, batch_idx, dataloader_idx)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 171, in evaluation_step
output = self.trainer.accelerator_backend.validation_step(args)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 73, in validation_step
return self._step(self.trainer.model.validation_step, args)
File "/datav/software/anaconda3/envs/py39th19/lib/python3.9/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 63, in _step
output = model_step(*args)
File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 321, in validation_step
return self._generative_step(batch)
File "/datav/my/multidoc2dial/scripts/rag/finetune_rag_dialdoc.py", line 363, in _generative_step
generated_ids = self.model.generate(
File "/datav/my/multidoc2dial/dialdoc/models/rag/modeling_rag_dialdoc.py", line 700, in generate
pre_processor = self._get_logits_processor(
TypeError: _get_logits_processor() missing 1 required positional argument: 'logits_processor'