search

hao-ai-lab / LookaheadDecoding

[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
https://arxiv.org/abs/2402.02057
Apache License 2.0
1.15k stars 67 forks source link

about verification branch #60

Open dhcode-cpp opened 6 months ago

dhcode-cpp commented 6 months ago

In this figure, have any difference blue token [1,2,3] and [1,2,3] in verification branch?

image

thanks for your reply.

Viol2000 commented 5 months ago

They are different candidates serached from the n-gram cache. Thanks for your pointing out. I will change the color to make it less confusing.