Error in seq2seq model

[helonin]

❔Question

使用seq2seq模型时，报错 “TypeError: tf__call() missing 1 required positional argument: 'v'” 经查，是seq2seq.py的第96行：attention_feature = self.attention(tf.expand_dims(prev_state[-1], 1), encoder_output) 引用layers.attention_layer的attention函数时缺少参数 v 麻烦大侠查看一下。

Additional context

[karta282950]

去attention_layer.py，修改下Attention里面的q,k,v就可以了

[LoneFireBlossom]

去attention_layer.py，修改下Attention里面的q,k,v就可以了

请问应该怎么改这几个参数啊，看了一天也没看懂🥲

[karta282950]

tfts/layers/attention_layer.py的Attention模块修改。

def call(self, q, v, k=None, mask=None): if k is None: k = v q = self.dense_q(q) k = self.dense_k(k) v = self.densev(v) q = tf.concat(tf.split(q, self.numheads, axis=2), axis=0) # multi-heads transfer to k = tf.concat(tf.split(k, self.numheads, axis=2), axis=0) v = tf.concat(tf.split(v, self.numheads, axis=2), axis=0) score = tf.linalg.matmul(q, k_, transpose_b=True) # => (batchheads) seq_q * seqk score /= tf.cast(tf.shape(q)[-1], tf.float32) * 0.5 if mask is not None: score = score tf.cast(mask, tf.float32) score = tf.nn.softmax(score) score = self.dropout(score) outputs = tf.linalg.matmul(score, v_) # (batchheads) seq_q * units outputs = tf.concat(tf.split(outputs, self.num_heads, axis=0), axis=2) return outputs

[LoneFireBlossom]

tfts/layers/attention_layer.py的Attention模块修改。

def call(self, q, v, k=None, mask=None): if k is None: k = v q = self.dense_q(q) k = self.dense_k(k) v = self.densev(v) q = tf.concat(tf.split(q, self.numheads, axis=2), axis=0) # multi-heads transfer to k = tf.concat(tf.split(k, self.numheads, axis=2), axis=0) v = tf.concat(tf.split(v, self.numheads, axis=2), axis=0) score = tf.linalg.matmul(q, k_, transpose_b=True) # => (batch_heads) seq_q seqk score /= tf.cast(tf.shape(q)[-1], tf.float32) * 0.5 if mask is not None: score = score tf.cast(mask, tf.float32) score = tf.nn.softmax(score) score = self.dropout(score) outputs = tf.linalg.matmul(score, v_) # (batch_heads) seq_q units outputs = tf.concat(tf.split(outputs, self.num_heads, axis=0), axis=2) return outputs

感谢，应该就是把参数改一下，然后前面加一个if：

def call(self, q, v, k=None, mask=None):
    if k is None:
        k = v
    ＃下面都不用改
    q = self.dense_q(q)  # project the query/key/value to num_heads * units
    k = self.dense_k(k)
    v = self.dense_v(v)

[karta282950]

是的, 我是参考tensorflow官网改的, 你可以看下：https://github.com/keras-team/keras/blob/v2.8.0/keras/layers/dense_attention.py#L221-L356 另外楼主如果有解决到问题, 麻烦把issue设为closed~