search

premAI-io / state-of-open-source-ai

:closed_book: Clarity in the current fast-paced mess of Open Source innovation
https://book.premai.io/state-of-open-source-ai
Other
1.52k stars 90 forks source link

Mixture of Experts (MoE) #122

Closed Anindyadeep closed 11 months ago

Anindyadeep commented 11 months ago

Type

new chapter

Chapter/Page

models

Description

Although MoE has been there in general Deep learning, just now Mistral Released their second models with MoE, so now it is very important to release those also.