I am trying the script in Running , and it get error like this
-- Process 3 terminated with the following error: Traceback (most recent call last): File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap fn(i, *args) File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/fairseq_cli/train.py", line 284, in distributed_main main(args, init_distributed=True) File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/fairseq_cli/train.py", line 68, in main extra_state, epoch_itr = checkpoint_utils.load_checkpoint(args, trainer) File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/fairseq/checkpoint_utils.py", line 109, in load_checkpoint reset_meters=args.reset_meters, File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/fairseq/trainer.py", line 173, in load_checkpoint 'Criterion does not match; please reset the optimizer (--reset-optimizer).' AssertionError: Criterion does not match; please reset the optimizer (--reset-optimizer).
And i check fairseq code in trainer.py error line
and here is my optimizer
criterion_name: MaskedLmLoss
self.criterion.class.name: LabelSmoothedCrossEntropyCriterion
i have no ideal
anyone help?
thank
I am trying the script in Running , and it get error like this
-- Process 3 terminated with the following error: Traceback (most recent call last): File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap fn(i, *args) File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/fairseq_cli/train.py", line 284, in distributed_main main(args, init_distributed=True) File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/fairseq_cli/train.py", line 68, in main extra_state, epoch_itr = checkpoint_utils.load_checkpoint(args, trainer) File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/fairseq/checkpoint_utils.py", line 109, in load_checkpoint reset_meters=args.reset_meters, File "/home/jimmyhsia0522/.local/lib/python3.7/site-packages/fairseq/trainer.py", line 173, in load_checkpoint 'Criterion does not match; please reset the optimizer (--reset-optimizer).' AssertionError: Criterion does not match; please reset the optimizer (--reset-optimizer).
And i check fairseq code in trainer.py error line and here is my optimizer criterion_name: MaskedLmLoss self.criterion.class.name: LabelSmoothedCrossEntropyCriterion i have no ideal anyone help? thank