Open cui-z opened 2 years ago
calculate attention_score `
outputs = tf.concat([output[0], output[1]], 2) # output[0] : lstm_fw, output[1] : lstm_bw outputs = tf.transpose(outputs, [1, 0, 2]) # [n_step, batch_size, n_hidden]
final_hidden_state = outputs[-1] output_all = tf.concat([output[0], output[1]], 2) final_hidden_state = tf.expand_dims(final_hidden_state, 2) attn_weights = tf.squeeze(tf.matmul(output_all, final_hidden_state), 2) `
calculate attention_score `
Attention
outputs = tf.concat([output[0], output[1]], 2) # output[0] : lstm_fw, output[1] : lstm_bw outputs = tf.transpose(outputs, [1, 0, 2]) # [n_step, batch_size, n_hidden]
只用了最后一个步长的输出
final_hidden_state = outputs[-1] output_all = tf.concat([output[0], output[1]], 2)
final_hidden_state = tf.expand_dims(final_hidden_state, 2)
attn_weights = tf.squeeze(tf.matmul(output_all, final_hidden_state), 2) `