lucidrains rotary-embedding-torch issues

lucidrains / rotary-embedding-torch

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

MIT License

573 stars 44 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Unused params if cache_if_possible=True when multiple rotary dimensions are used

#37 lukasschmit closed 1 month ago
1
Slower than absolute positional embeddings?

#36 umarbutler opened 2 months ago
4
Explicit casting instead of autocasting

#35 lminer opened 2 months ago
0
Fix chi scale multiplication

#34 TimFelixBeyer closed 2 months ago
1
Add sequence position interpolation to axial RoPE

#33 tasansal closed 2 months ago
1
Fine-tuning Axial RoPE with frequency scaling?

#32 tasansal opened 2 months ago
0
apply_rotary_emb - remove inplace operation

#31 blasscoc closed 3 months ago
1
RoPE embeddings

#30 PRamoneda opened 3 months ago
1
Request for YaRN

#29 VarunGumma opened 3 months ago
0
Lastest commit incompatible with local_attention

#28 MarcusLoppe closed 4 months ago
3
xPOS embeddings during inference

#27 VarunGumma closed 4 months ago
2
LieRE: Generalizing Rotary Position Encodings. Beats RoPE-mixed by large margin and is much faster (compute-wise)

#26 kabachuha closed 1 month ago
28
RoPE-Mixed: Improvement over Axial for n-D

#25 tasansal opened 4 months ago
1
nan loss when training in fp8 with transformer engine

#24 saurabh-kataria closed 4 months ago
1
Repeat order.

#23 AliYoussef97 closed 4 months ago
0
added indexing

#22 AlxSp closed 8 months ago
0
Request for permission to publish a Rust port of this python module

#21 Mekadrom closed 8 months ago
1
`torch.cat` failes in `apply_rotary_emb` when `freqs.shape[-1] == t.shape[-1]`, and `start_index = 0`

#20 mattaltberg opened 8 months ago
1
RoPE on Images

#19 aaprasad closed 9 months ago
1
caching frequency results in RuntimeError: Trying to backward through the graph a second time

#18 wren93 closed 6 months ago
3
Is 'broadcat' part of the API?

#17 rsxdalv closed 11 months ago
5
Error caused by tensor-type seq_len

#16 cmunna0052 closed 11 months ago
1
Model hangs on eval

#15 GarrettMerz opened 11 months ago
18
implementing on vision tranformers

#14 mukvnd closed 1 year ago
2
Does this library support 2D RoPE embeddings?

#13 logicchains closed 1 year ago
2
Support for sequence length ordering

#12 iiSeymour closed 1 year ago
7
Bug in cache

#11 N0r9st closed 1 year ago
1
Usage with x-transformers

#10 sonovice opened 1 year ago
4
Bfloat16 support for use_xpos=True

#9 rostro36 closed 1 year ago
8
Using with xpos causes NaNs after rotating Q, K

#8 andersonbcdefg closed 1 year ago
2
AttributeError: 'NoneType' object has no attribute 'to'

#7 yingzhao27 closed 1 year ago
3
freqs reference

#6 biirving opened 1 year ago
0
Tricks for training with RoPE? Specific initialisers for QK projections?

#5 thorinf opened 1 year ago
0
why dim of q be different from dim of RotaryEmbedding

#4 HiSultryMan opened 1 year ago
2
Length Extrapolatable Rotary Embeddings

#3 hugofloresgarcia opened 1 year ago
2
Custom position offset when rotating queries or keys

#2 krasserm closed 1 year ago
1
about axial rotary embeddings

#1 raindrop313 opened 2 years ago
0