lucidrains rotary-embedding-torch issues

lucidrains / rotary-embedding-torch

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

MIT License

535 stars 43 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Slower than absolute positional embeddings?

#36 umarbutler opened 2 weeks ago
3
Explicit casting instead of autocasting

#35 lminer opened 3 weeks ago
0
Fix chi scale multiplication

#34 TimFelixBeyer closed 4 weeks ago
1
Add sequence position interpolation to axial RoPE

#33 tasansal closed 1 month ago
1
Fine-tuning Axial RoPE with frequency scaling?

#32 tasansal opened 1 month ago
0
apply_rotary_emb - remove inplace operation

#31 blasscoc closed 1 month ago
1
RoPE embeddings

#30 PRamoneda opened 1 month ago
0
Request for YaRN

#29 VarunGumma opened 2 months ago
0
Lastest commit incompatible with local_attention

#28 MarcusLoppe closed 2 months ago
3
xPOS embeddings during inference

#27 VarunGumma closed 3 months ago
2
LieRE: Generalizing Rotary Position Encodings. Beats RoPE-mixed by large margin and is much faster (compute-wise)

#26 kabachuha opened 3 months ago
8
RoPE-Mixed: Improvement over Axial for n-D

#25 tasansal opened 3 months ago
1
nan loss when training in fp8 with transformer engine

#24 saurabh-kataria closed 3 months ago
1
Repeat order.

#23 AliYoussef97 closed 2 months ago
0
added indexing

#22 AlxSp closed 7 months ago
0
Request for permission to publish a Rust port of this python module

#21 Mekadrom closed 6 months ago
1
`torch.cat` failes in `apply_rotary_emb` when `freqs.shape[-1] == t.shape[-1]`, and `start_index = 0`

#20 mattaltberg opened 7 months ago
1
RoPE on Images

#19 aaprasad closed 8 months ago
1
caching frequency results in RuntimeError: Trying to backward through the graph a second time

#18 wren93 closed 4 months ago
3
Is 'broadcat' part of the API?

#17 rsxdalv closed 9 months ago
5
Error caused by tensor-type seq_len

#16 cmunna0052 closed 9 months ago
1
Model hangs on eval

#15 GarrettMerz opened 9 months ago
18
implementing on vision tranformers

#14 mukvnd closed 10 months ago
2
Does this library support 2D RoPE embeddings?

#13 logicchains closed 12 months ago
2
Support for sequence length ordering

#12 iiSeymour closed 1 year ago
7
Bug in cache

#11 N0r9st closed 1 year ago
1
Usage with x-transformers

#10 sonovice opened 1 year ago
4
Bfloat16 support for use_xpos=True

#9 rostro36 closed 1 year ago
8
Using with xpos causes NaNs after rotating Q, K

#8 andersonbcdefg closed 1 year ago
2
AttributeError: 'NoneType' object has no attribute 'to'

#7 yingzhao27 closed 1 year ago
3
freqs reference

#6 biirving opened 1 year ago
0
Tricks for training with RoPE? Specific initialisers for QK projections?

#5 thorinf opened 1 year ago
0
why dim of q be different from dim of RotaryEmbedding

#4 HiSultryMan opened 1 year ago
2
Length Extrapolatable Rotary Embeddings

#3 hugofloresgarcia opened 1 year ago
2
Custom position offset when rotating queries or keys

#2 krasserm closed 1 year ago
1
about axial rotary embeddings

#1 raindrop313 opened 2 years ago
0