Maybe I don't interpret the model(...)-function correctly, but I see the following:
While training you put the correct and wrong rocstories together into the decoder. They both go through the embedding + decoder and then into the sparse_softmax_cross_entropy-function.
This means, though, that the model also learns to generate wrong sentences, or am I missing something?
My intuition would be to set all masks to 0 for the wrong sentences?!
Maybe I don't interpret the model(...)-function correctly, but I see the following:
While training you put the correct and wrong rocstories together into the decoder. They both go through the embedding + decoder and then into the sparse_softmax_cross_entropy-function.
This means, though, that the model also learns to generate wrong sentences, or am I missing something?
My intuition would be to set all masks to 0 for the wrong sentences?!
Thanks and regards