search

SafeAILab / EAGLE

Official Implementation of EAGLE
https://arxiv.org/pdf/2406.16858
Apache License 2.0
622 stars 59 forks source link

Questions about the EAGLE parameters, not match the pytorch_model.bin #60

Closed DeclK closed 2 months ago

DeclK commented 2 months ago

In the readme, the EAGLE parameters is 0.24B, but in the huggingface repo, the pytorch_model.bin is 1.4GB, which is abnormal for a 0.24B size model, what is inside the .bin?

Liyuhui-12 commented 2 months ago

The draft model of EAGLE uses the embedding layer of the target model. It is not trained but is saved in the checkpoint. This checkpoint saves weights in fp32 format, with each parameter occupying 4 bytes.

DeclK commented 2 months ago

That makes sense! Thanks for the reply, closing the issue.