search

wasiahmad / PLBART

Official code of our work, Unified Pre-training for Program Understanding and Generation [NAACL 2021].
https://arxiv.org/abs/2103.06333
MIT License
186 stars 34 forks source link

Samples of predictions and answers are not equal, {len(preds)}: {len(gts)} #10

Closed tmarrakchi closed 3 years ago

tmarrakchi commented 3 years ago

Hi,

I am trying to run the "run.sh" file, but I have the following error:

fairseq-generate: error: argument --task: invalid choice: 'translation_from_pretrained_bart' (choose from 'translation', 'translation_lev', 'language_modeling', 'sentence_prediction', 'multilingual_translation', 'translation_moe', 'semisupervised_translation', 'sentence_ranking', 'multilingual_masked_lm', 'cross_lingual_lm', 'audio_pretraining', 'translation_from_pretrained_xlm', 'denoising', 'masked_lm', 'legacy_masked_lm') Traceback (most recent call last): File "/content/drive/MyDrive/Colab_Notebooks/PLBART-main/scripts/text_to_code/evaluator.py", line 46, in <module> main() File "/content/drive/MyDrive/Colab_Notebooks/PLBART-main/scripts/text_to_code/evaluator.py", line 29, in main assert len(preds) == len(gts), f"Samples of predictions and answers are not equal, {len(preds)}: {len(gts)}" AssertionError: Samples of predictions and answers are not equal, 0: 2000 Traceback (most recent call last): File "/content/drive/MyDrive/Colab_Notebooks/PLBART-main/evaluation/CodeBLEU/calc_code_bleu.py", line 34, in <module> assert len(hypothesis) == len(pre_references[i]) AssertionError

Any idea please to fix this ?

wasiahmad commented 3 years ago

What version of Fairseq are you using?

tmarrakchi commented 3 years ago

fairseq==0.9.0

wasiahmad commented 3 years ago

I can see the error is due to the invalid task choice "translation_from_pretrained_bart" which is actually not part of fairseq==0.9.0. I see in v0.10.0, the task was introduced (https://github.com/pytorch/fairseq/tree/v0.10.0/fairseq/tasks).

I am using fairseq 0.9.0 but probably a later commit which consists of the translation_from_pretrained_bart task. Can you please try using fairseq v0.10.0?