search

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://arxiv.org/pdf/2406.16858
Apache License 2.0
780 stars 79 forks source link

Can EAGLE be applied to any auto-regressive model? #41

Closed sam-sepiola closed 5 months ago

sam-sepiola commented 7 months ago

How can I use EAGLE with a custom auto-regressive model that's not supported ( yet!) by default?

Liyuhui-12 commented 7 months ago

Yes, EAGLE doesn't depend on the internal structure of LLM. When training EAGLE with a custom auto-regressive mode, it is important to adjust the instruction template accordingly.

thyywr759 commented 6 months ago

In the ea_model.py:from_pretrained(), you used KVLlamaForCausalLM and KVMixtralForCausalLM as the base_model classes.

If we were to use another model, such as qwen, as the base model, how would we change this part of the code

Liyuhui-12 commented 6 months ago

@thyywr759 We have updated the Readme, and you can find the guide here.