Support Mixtral-8x22b Instruct

[mcotdev]

Any chance of support for open-mixtral-8x22b Instruct?

[simonw]

Working on that now.

[simonw]

Might work already. I ran this:

llm mistral refresh

And it said:

Added models: open-mixtral-8x22b-2404, open-mixtral-8x22b
New list of models:
open-mistral-7b
mistral-tiny-2312
mistral-tiny
open-mixtral-8x7b
open-mixtral-8x22b
open-mixtral-8x22b-2404
mistral-small-2312
mistral-small
mistral-small-2402
mistral-small-latest
mistral-medium-latest
mistral-medium-2312
mistral-medium
mistral-large-latest
mistral-large-2402

And now:

llm models | grep mistral

Outputs:

Mistral: mistral/open-mistral-7b
Mistral: mistral/mistral-tiny-2312
Mistral: mistral/mistral-tiny (aliases: mistral-tiny)
Mistral: mistral/open-mixtral-8x7b
Mistral: mistral/open-mixtral-8x22b
Mistral: mistral/open-mixtral-8x22b-2404
Mistral: mistral/mistral-small-2312
Mistral: mistral/mistral-small (aliases: mistral-small)
Mistral: mistral/mistral-small-2402
Mistral: mistral/mistral-small-latest
Mistral: mistral/mistral-medium-latest
Mistral: mistral/mistral-medium-2312
Mistral: mistral/mistral-medium (aliases: mistral-medium)
Mistral: mistral/mistral-large-latest (aliases: mistral-large)
Mistral: mistral/mistral-large-2402

So this works:

llm -m mistral/open-mixtral-8x22b 'Say hello in Spanish, German and Italian'

Sure, here are the translations:

• In Spanish, "hello" is "hola".
• In German, "hello" is "hallo".
• In Italian, "hello" is "ciao".

These are the most common ways to say "hello" in these languages. However, there are many other ways to greet someone, depending on the context and time of day.

[simonw]

Fresh installations of this plugin should get the new model automatically, because the list of models is fetched and cached the first time the plugin is used: https://github.com/simonw/llm-mistral/blob/329730ad69d7a8a38d56571144da8aa70cf6205c/llm_mistral.py#L49-L56

[simonw]

Added those instructions to the 0.3.1 release notes: https://github.com/simonw/llm-mistral/releases/tag/0.3.1