[QUESTION]Using HELM tool for EE-model in Megatron format

pan-x-c / EE-LLM

EE-LLM is a framework for large-scale training and inference of early-exit (EE) large language models (LLMs).

Other

48 stars 5 forks source link

[QUESTION]Using HELM tool for EE-model in Megatron format #8

Closed BenWu11 closed 7 months ago

BenWu11 commented 7 months ago

Hi!

I'm attempting to utilize the HELM tool to benchmark the EE model as mentioned in the paper. Hence, I initially believe I need to convert the ckpt format into the Hugging Face format and then utilize the HELM tool to test the EE model. However, this approach seems impractical.

Could you provide some methods for using the HELM tool with the EE model in Megatron-LM format?" Thanks.

pan-x-c commented 7 months ago

HELM has a megatron-lm connector, you can try to modify this connector and add EE configurations