failed to reproduce given speedup results

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)

https://arxiv.org/pdf/2406.16858

Apache License 2.0

780 stars 79 forks source link

failed to reproduce given speedup results #25

Closed ZepinLi closed 8 months ago

ZepinLi commented 8 months ago

hi EAGLE team,

thank you for great work! we failed to acquire your speedup results with mi250(rocm). experimental outcomes of llama2-13b-chat and llama2-13b-chat are below,

are your results really dependent on hardware? or, are we lost something crucial?

best.

hongyanz commented 8 months ago

There have been people who can reproduce our claimed numbers. Without more details about your setup, we suspect perhaps you are running something else in the backend? That will significantly affect the speed.

https://github.com/SafeAILab/EAGLE/issues/5

alcholiclg commented 3 months ago

Can I ask you how you solved the problem of not being able to reproduce the speedup ratio? I recently encountered a similar problem.