search

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://arxiv.org/pdf/2406.16858
Apache License 2.0
816 stars 81 forks source link

Will smaller models be supported? #116

Closed Puvoka closed 2 months ago

Puvoka commented 2 months ago

For example, 1.1B tinyllama.

hongyanz commented 2 months ago

Speculative decoding uses a small model to draft for a large model. The 1.1B tinyllama is already a small model. So there is no need to speculatively decode this model.