bcaitech1 / p4-fr-soccer

p4-fr-soccer created by GitHub Classroom

1 stars 3 forks source link

MHA솔루션 적용 #54

Closed karim-moon closed 2 years ago

karim-moon commented 3 years ago

51 (rezero)솔루션 ASTER, SATRN 적용

ASTER

[x] ASTER AttentionCell의 Attend부분 MHA로 변환 (Layernorm 적용 X)
[ ] FeedForward 추가(Layernorm 적용 X)

SATRN

[x] TransformersEncodeLayer, TransformersDecodeLayer MHA 이후 Layernorm 삭제