TODO list

[yezhengmao1]

• [ ] auto hyperparameters search
• [ ] more rlhf methods support
• [ ] more model support
• [ ] the multimodal support
• [ ] auto-parallelizing
• [ ] better dispatcher and monitor

[mikecovlee]

I will migrate improvement from MoE-PEFT to m-LoRA step by step :)

[merlintang]

great