search

Mintplex-Labs / anything-llm

The all-in-one Desktop & Docker AI application with full RAG and AI Agent capabilities.
https://useanything.com
MIT License
17.16k stars 1.84k forks source link

Add mistralai/Mixtral-8x7B-Instruct-v0.1 #450

Closed andsty closed 6 months ago

andsty commented 6 months ago

Trying with mistralai/Mixtral-8x7B-Instruct-v0.1 it seems that the chat works fine. when i add a document in my workspace the responses are not related to the document i have added at all

timothycarambat commented 6 months ago

Are you sure that the context from a relevant document is being passed to the model? Additionally, not all models behave the same. Are you running this via LocalAI or LMStudio or otherwise?

lunamidori5 commented 6 months ago

Are you sure that the context from a relevant document is being passed to the model? Additionally, not all models behave the same. Are you running this via LocalAI or LMStudio or otherwise?

Localai as of right now does not support Mixtral, they are working on adding support for it.

lunamidori5 commented 6 months ago

@andsty Localai now supports Mixtral

timothycarambat commented 6 months ago

Closing this as wontfix since support for this would need to come from the provider itself (LocalAI, LMStudio or another provider) and not from AnythingLLM directly.

Since LocalAI supports this model now you can use that!