lucidrains ring-attention-pytorch issues - Githubissues

lucidrains / ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

MIT License

451 stars 26 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

It seems like this repo can combine with deepspeed-ulysses.

#18 foreverpiano opened 1 month ago
0
RingAttention exception from Triton

#17 GindaChen closed 1 month ago
5
Ring attention with hugging face or accelerate deep speed

#16 vkaul11 opened 2 months ago
0
Question about RoPE

#15 U-rara closed 2 months ago
1
Is the GPU being used?

#14 overseerlabs opened 2 months ago
1
Cross Attention variant?

#13 Xynonners opened 4 months ago
1
Use triton flash attn backwards

#12 lucidrains closed 5 months ago
0
striped causal version of `ring_flash_attn_cuda` is not working

#11 lucidrains closed 5 months ago
2
Connection closed by peer

#10 jkl375 closed 5 months ago
1
Working towards CUDA use with NCCL backend

#9 Damjan-Kalajdzievski closed 5 months ago
16
inference for open LLM

#8 LzhinFdu closed 5 months ago
0
prepare to use flash-attn package, only for non-striped ring attention

#7 lucidrains closed 6 months ago
0
I'm doing an image generation experiment, but my script outputs a json file, how do I train a Transformer model to generate a pixel representation of an image?

#6 win10ogod closed 6 months ago
1
A ring attention with flash attention kernel implementation

#4 zhuzilin closed 6 months ago
19
ValueError: Invalid expression '[ True]', must be integers

#3 kyegomez closed 6 months ago
7
8 A100S

#2 kyegomez closed 5 months ago
1
Comment about use of all gather

#1 NielsRogge closed 7 months ago
2