Added Static KV Caching for all Huggingface Models

MDK8888 / GPTFast

Accelerate your Hugging Face Transformers 7.6-9x. Native to Hugging Face and PyTorch.

Apache License 2.0

686 stars 65 forks source link

Closed MDK8888 closed 7 months ago