Added Static KV Caching for all Huggingface Models

MDK8888 / GPTFast

Accelerate your Hugging Face Transformers 7.6-9x. Native to Hugging Face and PyTorch.

Apache License 2.0

677 stars 64 forks source link

Closed MDK8888 closed 5 months ago