search

lucidrains / rotary-embedding-torch

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch
MIT License
573 stars 44 forks source link

AttributeError: 'NoneType' object has no attribute 'to' #7

Closed yingzhao27 closed 1 year ago

yingzhao27 commented 1 year ago

When runing test case,
AttributeError: 'NoneType' object has no attribute 'to'

lucidrains commented 1 year ago

can you show the full script you are running?

yingzhao27 commented 1 year ago

can you show the full script you are running?

hello, I just use the code you provide in ”Usage", that is: `import torch from rotary_embedding_torch import RotaryEmbedding

rotary_emb = RotaryEmbedding(dim = 32)

q = torch.randn(1, 8, 1024, 64) # queries - (batch, heads, seq len, dimension of head) k = torch.randn(1, 8, 1024, 64) # keys

q = rotary_emb.rotate_queries_or_keys(q) k = rotary_emb.rotate_queries_or_keys(k) print(q)

yingzhao27 commented 1 year ago

can you show the full script you are running?

hello, I just use the code you provide in ”Usage", that is: `import torch from rotary_embedding_torch import RotaryEmbedding

rotary_emb = RotaryEmbedding(dim = 32)

q = torch.randn(1, 8, 1024, 64) # queries - (batch, heads, seq len, dimension of head) k = torch.randn(1, 8, 1024, 64) # keys

q = rotary_emb.rotate_queries_or_keys(q) k = rotary_emb.rotate_queries_or_keys(k) print(q)

Now it's OK. Sorry to disturb.