mistralai / mistral-inference

Official inference library for Mistral models
https://mistral.ai/
Apache License 2.0
9.37k stars 817 forks source link

(question) moe for conversations #125

Open Tom-Neverwinter opened 5 months ago

Tom-Neverwinter commented 5 months ago

As we have moe models where it picks two of the expert models, can we apply this to conversations?

You have a history of conversations with descriptive titles, couldn't the ai pick relevant conversations as memory? Or make or change conversations at will to the appropriate conversation creating a sudo rag?

Add: hidden conversations, internal thought/dialogues.