search

SafeAILab / EAGLE

Official Implementation of EAGLE-1 (ICML'24) and EAGLE-2 (EMNLP'24)
https://arxiv.org/pdf/2406.16858
Apache License 2.0
780 stars 79 forks source link

Question about manual datasets & Some interests about details #30

Closed HackGiter closed 7 months ago

HackGiter commented 8 months ago

Thanks for your great job. I just have some curiosities about it.

First, would it be too big to store the hidden states directly as .ckpt file? How large it is. Second, could you provide ablation experiment about the cnet and decoding methods?

Liyuhui-12 commented 8 months ago

It depends on the amount of data used and the hidden_size of the target LLM. Using ShareGPT (about 68k conversations), a model with hidden_size=4096 requires 740GB of disk space. The ablation experiment can be found in our paper.