strange out of range

[dgai91]

Using TensorFlow backend. Traceback (most recent call last): File "C:/Users/LawLi/PycharmProjects/first/attention/keras_att.py", line 181, in l_att = AttLayer()(l_dense) File "D:\ProgramData\Anaconda3\lib\site-packages\keras\engine\topology.py", line 596, in call output = self.call(inputs, **kwargs) File "C:/Users/LawLi/PycharmProjects/first/attention/keras_att.py", line 165, in call eij = K.tanh(K.dot(x, self.W)) File "D:\ProgramData\Anaconda3\lib\site-packages\keras\backend\tensorflow_backend.py", line 968, in dot y_permute_dim = [y_permute_dim.pop(-2)] + y_permute_dim IndexError: pop index out of range

[dgai91]

this problem has been triggered when i complied the network.guys please help.thank you

[jhoh10]

I also have had this same problem with tensorflow backend.

[jhoh10]

I believe this is is caused by: https://github.com/fchollet/keras/issues/2153

[dgai91]

thank you @jhoh10,i will check it today

[jhoh10]

let me know if you find a solution to this problem!

[dgai91]

i have solved tuis

[dgai91]

i have solved this to change the backend to theano and i guess you can reimplement the k.dot for nd tensor as the theano performance.this may fit the tensorflow backend

[jhoh10]

when you sove the k.dot problem in tensorflow other problems arise elsewhere in the code. Easiest to just use theano as backend.

[AlexGidiotis]

For Tensorflow backend you can replace this line: K.dot(x, kernel) with: K.squeeze(K.dot(x, K.expand_dims(kernel)), axis=-1)

[Siddharthss500]

For people who are using Tensorflow backend and are still facing issues after implementing the change suggested by @AlexGidiotis , you can replace the function "call" in the "class AttLayer":

def call(self, x, mask=None):
        eij = K.tanh(K.dot(x, self.W))

        ai = K.exp(eij)
        weights = ai/K.sum(ai, axis=1).dimshuffle(0,'x')

        weighted_input = x*weights.dimshuffle(0,1,'x')
        return weighted_input.sum(axis=1)

with :

 def call(self, x, mask=None):
        eij = K.tanh(K.squeeze(K.dot(x, K.expand_dims(self.W)), axis=-1))

        ai = K.exp(eij)
        weights = ai/K.expand_dims(K.sum(ai, axis=1),1)

        weighted_input = x*K.expand_dims(weights,2)
        return K.sum(weighted_input, axis=1)

This worked for me!

[dupsys]

Thanks, guys. The solution work for me.

[dgai91]

@jhoh10 I had solved this issue from Siddharthss500's code.

Thank you everybody