Closed BenWu11 closed 6 months ago
Currently, huggingface does not support EE models, so EE-LLM checkpoints cannot be converted to huggingface format.
Thanks for your useful suggestion.
Excuse me.
Could you please expain the reason why huggingface does not support EE models more?
Is it that the primary challenge with converting Early-exit models to the Hugging Face format lies in the architectural differences and the specific mechanisms used by these models?
The architecture and inference method of EE models are different from standard GPT models. For now, huggingface doesn't have an architecture that compatible with EE, and also doesn't provide EE inference method. We are planning to provide a huggingface version, but it may take some time due to other work schedules.
Hello!
I'm a senior high student,and I would like to inquire if there are any methods available to convert a checkpoint from Megatron-LM into the Hugging Face format after training.
This is because when I attempted to run the provided Python code below, it did not convert the checkpoint into the correct dimension size for the Hugging Face format.Thanks.