remove useless code - Githubissues

feifeibear / long-context-attention

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

Apache License 2.0

350 stars 24 forks source link

Closed feifeibear closed 6 months ago

feifeibear commented 6 months ago

sync version (AsyncLongContextAttention set self._async_op = False)

23.500 iter/sec
async without stream (AsyncLongContextAttention set self._async_op = True, stream is None)

23.194 iter/sec

async with stream (AsyncLongContextAttention set self._async_op = True, stream not None)

23.223 iters/sec
LongContextAttention 26.255 iter/sec