search

OpenBioML / protein-lm-scaling

Other
54 stars 15 forks source link

GQA Attention #59

Closed RishikeshMagar closed 5 months ago

RishikeshMagar commented 8 months ago

Implemented GQA attention with the tests. Discussed with buttercutter and Leo Chen on Discord

Issue: https://github.com/OpenBioML/protein-lm-scaling/issues/52

RishikeshMagar commented 7 months ago

Hey @pascalnotin, Sorry about the delay in response. I know the project has gone through a reboot and I have been following the progress discord.

I have made the changes and it is on my fork but I am not entirely sure how the requested changes work, do I submit a new PR or just keep the same one.