search

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://arxiv.org/pdf/2406.16858
Apache License 2.0
780 stars 79 forks source link

Support Chinese-LLaMA-Alpaca-2? #15

Closed skykiseki closed 9 months ago

skykiseki commented 9 months ago

Nice work! I just wonder will EAGLE support Chinese-LLaMA-Alpaca-2 and will you show the paper in the future? ?

model url: https://github.com/ymcui/Chinese-LLaMA-Alpaca-2

Hope for your response, thx

skykiseki commented 9 months ago

And I want to know the difference between EAGLE and SpecInfer. I'm guessing that the tree verification part of the two is the same, right?

Liyuhui-12 commented 9 months ago

We will support more models in the future.

Yes, the tree verification part is the same. Our main contribution lies in the guessing and drafting part, which is different from SpecInfer. Also, the application scope of EAGLE is different from SpecInfer. EAGLE is not limited to distributed LLM inference or offloading-based LLM inference, and it can be used in more general situations.

skykiseki commented 9 months ago

We will support more models in the future.

Yes, the tree verification part is the same. Our main contribution lies in the guessing and drafting part, which is different from SpecInfer. Also, the application scope of EAGLE is different from SpecInfer. EAGLE is not limited to distributed LLM inference or offloading-based LLM inference, and it can be used in more general situations.

OK,Looking forward to the upcoming updates, thank you.