FasterDecoding / Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
https://sites.google.com/view/medusa-llm
Apache License 2.0
2.28k stars 154 forks source link

[bug] fix preprocess function #101

Open xiezipeng-ML opened 6 months ago

xiezipeng-ML commented 6 months ago

There is a slight bug in the preprocess function, causing no difference between the content of targets and input_ids. Relevant modifications are proposed here: https://github.com/FasterDecoding/Medusa/pull/83#discussion_r1582080343