search

jl749 / the-clean-transformer

pytorch-lightning과 wandb로 깔끔하게 구현해보는 트랜스포머
0 stars 0 forks source link

Decoder Multi-Head Attention #17

Open jl749 opened 2 years ago

jl749 commented 2 years ago

image image

Query = search input (youtube search) Key = best keys matching query (search results, vid titles) Value = corresponding value to Key (video contents)

jl749 commented 2 years ago

Decoder's Multi-Head Attention

given the previous information and context of the target (kor-->eng, target=kor, previous_info=eng) what is the best semantic to come after the current context?