search

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://arxiv.org/pdf/2406.16858
Apache License 2.0
780 stars 79 forks source link

A simple question regarding to the paper. #32

Closed w32zhong closed 8 months ago

w32zhong commented 8 months ago

Not sure if here is the right place to ask, but I want to check my understanding here.

In the end of section 4.1, I quote here:

EAGLE generates a tree-structured draft. To enhance efficiency, we implement tree attention,
enabling the creation of a draft tree with a depth of m through m forward passes,
thereby encompassing more than m tokens.

Should it be "enabling the creation of a draft tree with a depth of m through 1 forward passes"? Should the point of draft tree be completing a tree of guess predictions in one forward pass?

Thanks!

w32zhong commented 8 months ago

Sorry, ignore my stupid question here. I guess it is because that section I am reading is the drafting process, not the verification process. Issue closed.