search

Infini-AI-Lab / Sequoia

scalable and robust tree-based speculative decoding algorithm
282 stars 29 forks source link

Is there any benchmark that compares Sequoia against vanilla speculative decoding? #10

Closed KexinFeng closed 3 months ago

KexinFeng commented 3 months ago

Hi,

Thanks for the great work!

I'm wondering if there is any benchmark that compares Sequoia against vanilla speculative decoding?

preminstrel commented 3 months ago

Once you get your acceptance rate, I think you can directly calculate the theoretical best gamma and speedup for vanilla speculative decoding.

dreaming-panda commented 3 months ago

Hello, you can use 4-chain, 8-chain in L40_growmaps as growmaps, which will produce "tree structures" for vanilla speculative decoding. Thank you!