warm-starting encoder-decoder models using EncoderDecoderModel always giving an empty string after fine-tuning

[kawsar-pie]

System Info

Hi @ArthurZucker and @patrickvonplaten, I am trying to train a seq2seq model using EncoderDecoderModel class and found this blog very helpful. Thanks to @patrickvonplaten for the nice explanation. Following this blog I fine-tuned a seq2seq model where I used a BERT (BanglaBERT an Electra) model as encoder and XGLM as decoder using BanglaParaphrase data. But after fine-tuning the model always generates an empty string. Now I do not understand where the problem is. Please help me to find the bug in the code.

Thanks.

Who can help?

@ArthurZucker @patrickvonplaten

Information

• [ ] The official example scripts
• [X] My own modified scripts

Tasks

• [x] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

Here is my notebook

Expected behavior

Input-output for my code: {'target': 'সিপিও আহত থাকায় যুদ্ধ পরিচালনার দায়িত্ব এসে পড়েছিল সেম্প্রোনিয়াসের কাঁধে।', 'pred_target': ''}

which should be something like this (should give the paraphrased sentence according to the input sentence in Bangla): {'target': 'সিপিও আহত থাকায় যুদ্ধ পরিচালনার দায়িত্ব এসে পড়েছিল সেম্প্রোনিয়াসের কাঁধে।', 'pred_target': 'সিপিও কর্তৃক আহত হয়ে সেমপ্রোনিয়াসের কাঁধে যুদ্ধ পরিচালনার দায়িত্ব আসে।'}

[amyeroberts]

Hi @kawsar-pie, thanks for raising an issue!

This is a question best placed in our forums. We try to reserve the github issues for feature requests and bug reports.

[kawsar-pie]

Thank you @amyeroberts for your comment. I created a new topic in the forum for this issue. I hope to find a solution from there.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.