search

FasterDecoding / Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
https://sites.google.com/view/medusa-llm
Apache License 2.0
2.28k stars 155 forks source link

[New feature] Fine-tune Medusa heads during SFT #36

Closed ctlllll closed 9 months ago

santhosh97 commented 1 year ago

What does this exactly mean? Does this mean training all the parameters including the original model (unfreezing the original model) parameters + medusa heads from scratch?

ctlllll commented 1 year ago

What does this exactly mean? Does this mean training all the parameters including the original model (unfreezing the original model) parameters + medusa heads from scratch?

Ah yes. I'll make it clearer, thanks!

ctlllll commented 1 year ago

Track: I added initial support for Medusa to the popular fine-tuning codebase. See commit: https://github.com/ctlllll/axolotl

ctlllll commented 9 months ago

Added in v1.0.