search

lucidrains / x-transformers

A simple but complete full-attention transformer with a set of promising experimental features from various papers
MIT License
4.42k stars 377 forks source link

Fix xpos when using mems #234

Closed pfeatherstone closed 6 months ago

lucidrains commented 6 months ago

@pfeatherstone hey, this looks good! thank you for the PR

pfeatherstone commented 6 months ago

oops. there was a typo. i used t.typ_as(self.inv_freq) instead of t.type_as(self.inv_freq)

lucidrains commented 6 months ago

no biggie, fixed