search

axolotl-ai-cloud / axolotl

Go ahead and axolotl questions
https://axolotl-ai-cloud.github.io/axolotl/
Apache License 2.0
7.85k stars 864 forks source link

set z3 leaf for deepseek v2 #1809

Closed winglian closed 2 months ago

winglian commented 2 months ago

need to include deepseek v2 as an moe arch for deepspeed zero3