How to optimize the scene with relatively short output?

hao-ai-lab / LookaheadDecoding

Apache License 2.0

1.04k stars 63 forks source link

How to optimize the scene with relatively short output? #63

Open yangbohust opened 1 month ago

yangbohust commented 1 month ago

If the output is relatively short, the acceleration effect of lookahead is relatively poor. How to improve the acceleration performance when the output is short?