feat: add support for flash_attn>=2.6.0

feifeibear / long-context-attention

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

Apache License 2.0

350 stars 24 forks source link

Closed Eigensystem closed 2 months ago