search

AbrahamSanders / seq2seq-chatbot

A sequence2sequence chatbot implementation with TensorFlow.
MIT License
99 stars 56 forks source link

Question from Deep NLP A-Z course #3

Closed lionely closed 6 years ago

lionely commented 6 years ago

How did you convert this section where we need the following values: attention_keys, attention_values ,attention_score_function,attention_construction_function

to work for a version of TensorFlow > 1.0 I can try to give more detail if you need it. The course instructor said your repo would be helpful, but it doesn't resemble the code in the course at all anymore. It's hard to know exactly what you changed to fix the issues. I don't understand the logic of which function replaces this in the new API. Please help!

AbrahamSanders commented 6 years ago

@lionely Hey - I sincerely apologize I did not see this sooner. I had my GitHub notification settings messed up (fixed now!).

The divergence you saw from the original course is that the TensorFlow seq2seq module was largely rewritten and broke compatibility since TF 1.2. I adapted my implementation using the Google NMT example on Tensorflow's website which clearly showed how the new seq2seq module is used. See here for the tutorial.

The attention functions were replaced by the AttentionWrapper class which wraps the LSTM cell to keep the same interface - making it very easy to pass either an attentional or non-attentional cell into the BasicDecoder or BeamSearchDecoder objects.