search

hunkim / word-rnn-tensorflow

Multi-layer Recurrent Neural Networks (LSTM, RNN) for word-level language models in Python using TensorFlow.
MIT License
1.31k stars 495 forks source link

Beam search? #28

Open hunkim opened 7 years ago

hunkim commented 7 years ago

Currently, we are using weighted_pick to select outputs:

def weighted_pick(weights):
            t = np.cumsum(weights)
            s = np.sum(weights)
            return(int(np.searchsorted(t, np.random.rand(1)*s)))

Should we also add beam search as an option?

See beam search implementations in Tensorflow https://github.com/tensorflow/tensorflow/issues/654!

Rojina99 commented 6 years ago

Can you please explain why cumulative sum is used here ?