Position Embedding inTransformer

[SilenceWinter]

a2_transformer_classification.py

1.1 position embedding for encoder input

    input_x_embeded = tf.nn.embedding_lookup(self.Embedding,self.input_x)  #[None,sequence_length, embed_size]
    input_x_embeded=tf.multiply(input_x_embeded,tf.sqrt(tf.cast(self.d_model,dtype=tf.float32)))
    **input_mask=tf.get_variable("input_mask",[self.sequence_length,1],initializer=self.initializer)**
    input_x_embeded=tf.add(input_x_embeded,input_mask) #[None,sequence_length,embed_size].position embedding.

这里的位置向量和论文中提出的Position Embedding不同，想问下，您这么写的区别在哪里呢？

[brightmart]

although we can follow idea from paper to get positional embedding, but in fact, positional embedding can be learned. it can be found in recurrent entity network, or some other paper. attention is all you need also mentioned that you can learned positional embedding similar like above.