search

PKU-MARL / Multi-Agent-Transformer

323 stars 59 forks source link

details about the decoder architecture #35

Open geruome opened 4 months ago

geruome commented 4 months ago

Hi, I am very interested in this series of work and have a question about the decoder architecture. I have noticed that the decoder architecture is a little bit different from which in the original Transformer paper.

1714382902050

image-15 The differences are circled in red. The K in QKV of second attention block and the skip connection are from encoder in our methods, while from decoder originally. Is it any improvement of the transformer architecture from some existing papers or just our design？