LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers - Githubissues

pentium3 / sys_reading

system paper reading notes

235 stars 12 forks source link

LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers #295

Open pentium3 opened 1 year ago

pentium3 commented 1 year ago

https://arxiv.org/pdf/2310.03294.pdf

pentium3 commented 1 year ago

https://x.com/rulinshao/status/1711836608742437159?s=46