zhougr1993 / DeepInterestNetwork

1.62k stars 558 forks source link

关于attention的实现 #50

Closed ZhengYukai666 closed 4 years ago

ZhengYukai666 commented 5 years ago

问题1:论文中attention里有一个out product是叉乘/外积的意思吗?

问题2:代码中的实现方法是tf.concat([queries, keys, queries-keys, queries*keys], axis=-1) 但是论文中是将queries, keys, out product(queries, keys)这三个东西concat起来,为什么代码和论文不一致呢?

ZhangYuef commented 4 years ago

问题1:论文中attention里有一个out product是叉乘/外积的意思吗?

问题2:代码中的实现方法是tf.concat([queries, keys, queries-keys, queries*keys], axis=-1) 但是论文中是将queries, keys, out product(queries, keys)这三个东西concat起来,为什么代码和论文不一致呢?

ISSUE #8 #23 对这个问题有所解答,可参考~