Too many UNK in the output

[LeenaShekhar]

I trained the network for 5000 iterations; see the loss below

Building Graph Training started

Model saved to disk at iteration #1000 val loss : 3.290424

Model saved to disk at iteration #2000 val loss : 3.261373

Model saved to disk at iteration #3000 val loss : 3.224990

Model saved to disk at iteration #4000 val loss : 3.151570

Model saved to disk at iteration #5000 val loss : 3.155647

After this I wanted to evaluate the model on the test dataset. Most of the decoder's output is "unk" (see below):

q : [hillary is crazy also evil nothing good about her except that she has a terminal illness]; a : [i unk unk unk unk unk unk unk unk unk unk unk unk unk] q : [breaking unk unk israeli unk and unk peace prize winner dies at unk]; a : [unk unk unk unk unk unk unk unk unk unk unk unk unk] q : [because and jason unk are fighting in the cage next week to see who unk into whom]; a : [i unk unk unk unk unk unk unk unk unk unk unk unk unk] q : [im considering unk a ticket shit looks live ]; a : [i unk unk unk] q : [unk is a classic but tears in heaven is stupid]; a : [i unk unk unk unk unk unk unk unk unk unk unk unk unk]

Do you think the output is like that because I tested the model's performance too soon or that the model is not learning anything?

[karanpande]

I am facing with the exact same problem, can someone guide us.

[LeenaShekhar]

I have not worked on it after that, but from experience it might be because of the training. Usually you need t train for longer steps in such models. Try that and see it that works. I have not explored the pre-trained models tough. You can compare your output with that.