feat: usp (unified sequence parallelism)

jzhang38 / EasyContext

Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.

Apache License 2.0

650 stars 47 forks source link

Closed feifeibear closed 2 months ago