Added primitives for speculative decoding and tests

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

https://petals.dev

MIT License

9.25k stars 525 forks source link

Closed xtinkt closed 3 months ago