search

krushi1992 / MOA-transformer

50 stars 7 forks source link

code #9

Open punk1 opened 1 year ago

punk1 commented 1 year ago

https://github.com/krushi1992/MOA-transformer/blob/61b98df3652b7d70d44e0ac22991baa28c14c632/models/MOA_transformer.py#L256

Why k v and q have the same shape

punk1 commented 1 year ago

There will be problems running, the shape is wrong