Thanks for your excellent work.I am confused about the target input_ids mentioned above /eagle/train/main.py .It seems that the target input_ids are sourced from dataset instead of the target LLM.
But I find that function getkacc() calculates the draft token acc with the hidden_states of target input_ids directly. For example ,
the dataset input_ids is "Deep learning is a powerful approach to machine learning" ,
the orginal LLM output is " Deep learning is a subfield of machine learning" .
so the target input_ids in dataset is calculated as follows:
"Deep learning is a powerful approach to machine learning" -> "learning is a subfield approach to machine learning"(target input_ids).
It should be noted that the next token of "Deep learning is a powerful" in original LLM is "approach", hypothetically.
the EAGLE draft prediction is as follows in getkacc() :
"Deep learning is a"-> "subfield"
"Deep learning is a subfield"-> "approach"
In fact , the real EAGLE inference in original LLM is as follows:
"Deep learning is a"-> "subfield"
"Deep learning is a subfield"-> "of"
I would like to know whether there's an issue with the getkacc() function calculation or if my understanding is flawed.
the EAGLE draft prediction is as follows in getkacc() : "Deep learning is a"-> "subfield" "Deep learning is a subfield"-> "approach"
In fact , the real EAGLE inference in original LLM is as follows: "Deep learning is a"-> "subfield" "Deep learning is a subfield"-> "of"
I would like to know whether there's an issue with the getkacc() function calculation or if my understanding is flawed.