search

jadore801120 / attention-is-all-you-need-pytorch

A PyTorch implementation of the Transformer model in "Attention is All You Need".
MIT License
8.78k stars 1.97k forks source link

About target mask #217

Open KimRass opened 8 months ago

KimRass commented 8 months ago

https://github.com/jadore801120/attention-is-all-you-need-pytorch/blob/master/transformer/Models.py#L190

I think that should be like

trg_mask = get_pad_mask(trg_seq, self.trg_pad_idx) | get_subsequent_mask(trg_seq)

am I wrong?