search

SafeAILab / EAGLE

Official Implementation of EAGLE
https://arxiv.org/pdf/2406.16858
Apache License 2.0
622 stars 59 forks source link

Question about alpha #80

Open zkqq opened 3 weeks ago

zkqq commented 3 weeks ago

Thank you very much for your work on EAGLE; it has been extremely helpful to me.

Could you please clarify what alpha means in this context? Does it refer to the accuracy of the EAGLE draft head in autoregressively predicting the next n tokens without using tree attention?

I look forward to your response. Thank you.

Liyuhui-12 commented 3 weeks ago

Does it refer to the accuracy of the EAGLE draft head in autoregressively predicting the next n tokens without using tree attention?

Yes. The input of the draft model is a feature sequence, part of which comes from the base model without errors, and part of which comes from the predictions of the draft model with errors. The acceptance rate of inputting n erroneous features is n-alpha.

zkqq commented 3 weeks ago

Does it refer to the accuracy of the EAGLE draft head in autoregressively predicting the next n tokens without using tree attention?

Yes. The input of the draft model is a feature sequence, part of which comes from the base model without errors, and part of which comes from the predictions of the draft model with errors. The acceptance rate of inputting n erroneous features is n-alpha.

Thank you very much for your response; it has been extremely helpful to me.

I have a follow-up question regarding the term alpha_num. Could you please explain its meaning? Additionally, why is it necessary to divide alpha by alpha_num when calculating the accuracy of the draft head, rather than dividing alpha by the number of drafts?

I look forward to your clarification. Thank you once again for your assistance.

Liyuhui-12 commented 3 days ago

image