Seperate prefilling and decoding for increamental decoding

flexflow / FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving

https://flexflow.readthedocs.io

Apache License 2.0

1.67k stars 224 forks source link

Closed zwang86 closed 6 months ago

zwang86 commented 6 months ago

Description of changes:

Related Issues:

Linked Issues:

Issues closed by this PR:

This change is