More advanced support for MoE finetune

Alpha-VLLM / LLaMA2-Accessory

An Open-source Toolkit for LLM Development

https://llama2-accessory.readthedocs.io/

Other

2.61k stars 167 forks source link

Open Jack47 opened 2 months ago

Jack47 commented 2 months ago

It's exciting to see

“More advanced MoE support will come soon.”

May I know what aspects of advanced MoE support you guys want to support?