search

databricks / megablocks

Apache License 2.0
1.11k stars 154 forks source link

Add Shared Expert #109

Closed vchiley closed 1 month ago

vchiley commented 1 month ago

Add Shared Expert similar to the DeepSeekMoE impl