Is your b_ij wrong?

[junfanlin]

Hi, Thanks for your contribution. And I think the b_ij defined in your code probably is unmatched with the paper. Your code is:

self.b_ij = tf.get_variable('b_ij', shape=(1, 1152, 1, 1))
...
c_i = tf.nn.softmax(self.b_ij, dim=1)

But in fact it should be

b_i = tf.get_variable(‘b_i’, shape=(1, 1152, 16, 1))
...
c_i = tf.nn.softmax(b_i, dim=2)

If I have misunderstand your code, please ignore me. Thanks~

[naturomics]

@junfanlin Indeed, there are some problems including the one you found, I'm fixing it, thanks for your reminding.