search

localminimum / R-net

A Tensorflow Implementation of R-net: Machine reading comprehension with self matching networks
MIT License
323 stars 122 forks source link

some problem in pointer_net #8

Closed kelvinleen closed 7 years ago

kelvinleen commented 7 years ago

in layers.py function pointer_net, i have some confusion in compute attention_pool: p1_logits = attention(inputs, Params.attn_size, weights_p, output_fn = None, scope = "attention") scores = tf.expand_dims(p1_logits, -1) scores = tf.nn.softmax(scores)

i think the softmax should compute before expand_dims, otherwise the softmax computes on the last dim which just one element makes score all 1s.

is anything i misunderstand?

ghost commented 7 years ago

Very nice catch! Thank you! Do you want to submit a pull request? :)

becxer commented 7 years ago

Yes, this is same as what I found today! softmax before expand will makes all variable in matrix goes to 1.

kelvinleen commented 7 years ago

thank u for reply so quickly. just submit a pull request. BTW: source code have change a lot compared to 2-weeks ago version. thx for ur code