search

udibr / headlines

Automatically generate headlines to short articles
MIT License
526 stars 150 forks source link

Bidirectional LSTM #24

Open TinaB19 opened 7 years ago

TinaB19 commented 7 years ago

I tried to use bidirectional lstm with merge_mode='sum' for encoding, but when I try to predict headlines, the model barely generates anything; However, the loss is lower than when I use the simple lstm. This is the only change that I made. Do you know why this happens?

udibr commented 7 years ago

generating with BiLSTM is tricky. In generation you move forward step by step generating one word at a time. However the BiLSTM model requires you to move both forward and backward.

If you want to use BiLSTM you can use it on the steps that are running over the article content and then run just a forward LSTM on the headline so latter you can generate your own headline from this forward only pass

TinaB19 commented 7 years ago

Can this be done by changing the current model? Considering that you feed both description and headline as input to a same model? I tried to do it myself but had no luck.