search

SafeAILab / EAGLE

Official Implementation of EAGLE
https://arxiv.org/pdf/2406.16858
Apache License 2.0
622 stars 59 forks source link

How many decoder layer needs to add in EAGLE? #46

Closed yjdy closed 2 months ago

yjdy commented 4 months ago

Thanks for the great work. I find a different between code and paper

In the paper, it say eagle trains a decoder layer. In the code, cnet.py class Model (https://github.com/SafeAILab/EAGLE/blob/main/model/cnets.py#L491)

self.layers = nn.ModuleList([LlamaDecoderLayer(config, index) for index in range(config.num_hidden_layers)])

It needs to add a decoder layer for every hidden layer.

Please tell me which one is correct.

Best regards

cyLi-Tiger commented 4 months ago

This config is not the config file in your llm directory, check the config.json in your ea_model_path, config.num_hidden_layers should be 1.

yjdy commented 4 months ago

Thanks for the response. If I need to train eagle on different LLM, I think I should prepare different config file. I just need to copy the config from the LLM I want to train on and change the num_hidden_layers to 1, right?

cyLi-Tiger commented 4 months ago

Not 100% sure, you better check the code in train/main.py

Liyuhui-12 commented 4 months ago

If I need to train eagle on different LLM, I think I should prepare different config file. I just need to copy the config from the LLM I want to train on and change the num_hidden_layers to 1, right?

You are right.

Liyuhui-12 commented 4 months ago

I find a different between code and paper.

Thank you for your interest. As @cyLi-Tiger mentioned, the configuration of EAGLE is different from the base LLM.