hao-ai-lab / LookaheadDecoding

Apache License 2.0
1.04k stars 63 forks source link

question on verification beginning pos #52

Open felixdae opened 3 months ago

felixdae commented 3 months ago

from this figure, we know green 1 is conditioned on blue 0, so it is the final true token. therefore since red 2 is conditioned on green 1 and blue 0, red 2 should also the correct token.

now we already have correct token on position 1 and 2, we should verify ngram start from 2. why we only verify ngram start from 0, isn't it a waste?

Viol2000 commented 1 month ago

Only Blue 0 is the final true token. orange/green/red tokens are tokens in a sliding window that will continuously generate new n-gram speculations.