Closed upskyy closed 1 year ago
Fixes #176
I referred to fairseq's conformer layer multi-head attention. [code] Also, I confirmed that it is training.
_relative_shift
B X n_head X T X 2T-1
B X n_head X T X T
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
What does this PR do?
Fixes #176
I referred to fairseq's conformer layer multi-head attention. [code] Also, I confirmed that it is training.
_relative_shift
methodB X n_head X T X 2T-1
B X n_head X T X T
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag