lucidrains / FLASH-pytorch

Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"
MIT License
344 stars 24 forks source link

About negative values in my input sentence embeddings #8

Open justinwoo97 opened 1 year ago

justinwoo97 commented 1 year ago

Does FLASHTransformer accept negative values in sentence embedding? I passed the BERT sentence embedding into the FLASHTransformer, and got the following error:

IndexError: index out of range in self

I then tried the README method: x = torch.randint(0, 20000, (1, 1024)) <-- works fine x = torch.randint(-1, 1, (1, 1024)) <-- same error

Thx