search

hao-ai-lab / LookaheadDecoding

[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
https://arxiv.org/abs/2402.02057
Apache License 2.0
1.15k stars 67 forks source link

Any plan to support other LLMs, and integrated into huggingface? #39

Closed henryxiao1997 closed 11 months ago

henryxiao1997 commented 11 months ago

Do you have the plan to support Mistral, Sola, Falcon (licence friendly models)? Roughly the timeline?

Also, do you plan to integrate into huggingface to benefit broader audiences?

Keen to see that!

Thanks!

Viol2000 commented 11 months ago

Yes, we plan to support other LLMs, and I think it will not take long. We are also planning to integrate it into huggingface.

henryxiao1997 commented 11 months ago

That's great! Thank you!