search

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://arxiv.org/pdf/2406.16858
Apache License 2.0
780 stars 79 forks source link

Difference between Eagle and SpecInfer #2

Closed Hsword closed 9 months ago

Hsword commented 9 months ago

Hi, Eagle team, I am one of the authors of SpecInfer (https://arxiv.org/pdf/2305.09781.pdf). I just noticed your work on exploring speculative inference of LLMs. I am curious about some contents in your blog (https://sites.google.com/view/eagle-llm, such as token tree structure and multi-round speculative sampling), what's the difference compared with the token tree design and multi-step speculative sampling in our SpecInfer paper? As you mentioned Medusa, they referred our work to respect our contributions about tree attention (https://sites.google.com/view/medusa-llm).

Liyuhui-12 commented 9 months ago

Thank you for the reminder. The validation part of EAGLE is the same as SpecInfer. Their difference lies in the guessing part.

haiduo commented 2 months ago

Hi, Eagle team, I am one of the authors of SpecInfer (https://arxiv.org/pdf/2305.09781.pdf). I just noticed your work on exploring speculative inference of LLMs. I am curious about some contents in your blog (https://sites.google.com/view/eagle-llm, such as token tree structure and multi-round speculative sampling), what's the difference compared with the token tree design and multi-step speculative sampling in our SpecInfer paper? As you mentioned Medusa, they referred our work to respect our contributions about tree attention (https://sites.google.com/view/medusa-llm).

Thanks for your work, but I'm curious if there are other implementations of SpecInfer that strip out the FlexFlow system, such as a pure Python version?