search

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://arxiv.org/pdf/2406.16858
Apache License 2.0
801 stars 80 forks source link

Question regarding sampling technique in EAGLE papers #148

Open llsj14 opened 4 days ago

llsj14 commented 4 days ago

Hello,

Thank you for your excellent work on EAGLE and EAGLE-2. I have been reading both papers, as well as Medusa.

The accept rate and speed-up metrics from the various experiments in the paper are valuable. However, I have a question regarding the sampling method you used. I’m curious whether you applied rejection sampling or the typical acceptance scheme proposed by Medusa.

The papers typically show a 10-15% difference in accept rate, aside from the sampling method. It would be helpful to understand which method you used in your experiments.

Thank you

Liyuhui-12 commented 14 hours ago

Thank you for your interest. We employ strict speculative sampling to ensure lossless results, rather than using the typical acceptance scheme found in Medusa.