Additional assert statement for dims alignment

yandexdataschool / nlp_course

YSDA course in Natural Language Processing

https://lena-voita.github.io/nlp_course.html

MIT License

9.83k stars 2.61k forks source link

Additional assert statement for dims alignment #131

Open recycletechno opened 1 year ago

recycletechno commented 1 year ago

Propose adding an additional assert statement in the MaskedSelfAttention class to verify that the number of attention heads matches the dim size. Otherwise, if self.dim is not divisible by self.num_heads without a remainder, there may be subtle and hard-to-detect errors inside nn.Module() when calling the TransformerLayer class in the future.

review-notebook-app[bot] commented 1 year ago

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.