search

tensorflow / models

Models and examples built with TensorFlow
Other
77.18k stars 45.75k forks source link

textsum::outputs are always the same with abstracts #507

Closed feifeipeng closed 8 years ago

feifeipeng commented 8 years ago

Hi guys,

I trained the model using the given toy data set and the given parameters. And when I tried to test with just one article from the toy dataset and with the example. The decoder is always the same with the given abstract.

For example, string of test-0 is article=<d> <p> <s>novell inc chief executive officer eric schmidt has been named chairman of the internet search-engine company google . </s></p></d> abstract=<d> <p> <s> novell ceo named google chairman. </s> </p> </d> and the outputs are always output=novell ceo named google chairman.

Why is this?

xtr33me commented 8 years ago

In practice, you should not be testing with the same data you trained with. Usually it is best if you take 30% of your data and set aside for testing. That said, if you run the model passing in let's say 10 of the articles, you will see a bit more variation probably. When I ran against the toy dataset, I sometimes saw what you are describing, but the majority of the time that was not the case. To be honest, if you had a model that was able to generate the expected abstract title with the articles sent in, in your 30% test data...you would have a pretty perfect model :)

tatatodd commented 8 years ago

@feifeipeng I think @xtr33me has given a great explanation. I'm closing this out, but feel free to comment on this issue or file a new issue if you have other problems.