Any plan to support other LLMs, and integrated into huggingface?

hao-ai-lab / LookaheadDecoding

[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

https://arxiv.org/abs/2402.02057

Apache License 2.0

1.11k stars 65 forks source link

Closed henryxiao1997 closed 9 months ago

henryxiao1997 commented 9 months ago

Do you have the plan to support Mistral, Sola, Falcon (licence friendly models)? Roughly the timeline?

Also, do you plan to integrate into huggingface to benefit broader audiences?

Keen to see that!

Thanks!

Viol2000 commented 9 months ago

Yes, we plan to support other LLMs, and I think it will not take long. We are also planning to integrate it into huggingface.

henryxiao1997 commented 9 months ago

That's great! Thank you!