Is this code work?

[hansd410]

Hi, I am not sure the code is working properly.

After training 20 epochs, generating result comes like:

[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !
[CLS] wann mochten sie im hotel innsbruck ubernachten ? [SEP] ubernachten ? [SEP] familien ubernachten ! [SEP] ! [SEP] !

Have you ever generated based on trained model?

[utkd]

Hi, yes I had generated using the model on this and a few other datasets. This is a frequent problem with text generation models and I was able to fix in most cases by changing the hyperparameters and other knobs or cleaning the data. Unfortunately, it has been more than a year and I do not have my original workspace to find the output. I will try to re-run this code and let you know if I see a similar result.

[Bk073]

@hansd410 how did you run the inference? During the inference, the input to the decoder will be the embedding from the encoder and start token, in that case how can we run the inference ?