Closed jackmpcollins closed 6 months ago
As of https://github.com/jackmpcollins/magentic/releases/tag/v0.19.0 , Mistral works via the openai API as long as the return type is a string type or a union of types (including simply making the return type nullable with | None
). Single type return types are broken.
Working example
from magentic import prompt, OpenaiChatModel
from magentic.chat_model.litellm_chat_model import LitellmChatModel
@prompt(
"""Use the tool to return a list of fruits.
Do not say anthing else. Just use the tool directly.
""",
model=OpenaiChatModel(
"mistral-large-latest",
api_key="<API_KEY>",
base_url="https://api.mistral.ai/v1/"
),
)
async def test() -> list[str] | None: ...
await test()
Added mistral backend with MistralChatModel
in https://github.com/jackmpcollins/magentic/releases/tag/v0.21.0
Example
from magentic import prompt
from magentic.chat_model.mistral_chat_model import MistralChatModel
from pydantic import BaseModel
class Superhero(BaseModel):
name: str
age: int
power: str
enemies: list[str]
@prompt(
"""Create a Superhero named {name}""",
model=MistralChatModel("mistral-large-latest"),
)
def create_superhero(name: str) -> Superhero: ...
create_superhero("Garden Man")
# Superhero(name='Garden Man', age=35, power='Plant control', enemies=['Smog', 'Deforestator'])
Ideally this can be done by simply setting
api_base
inOpenaiChatModel
. That avoids any new dependencies, and basically any new code. This is currently blocked on the Mistral API having a different format fortool_choice
than the OpenAI API. Issue for this https://github.com/mistralai/client-python/issues/78If that issue is not resolved, a new
MistralChatModel
can be added that reuses all the code fromOpenaiChatModel
but changes the defaultapi_base
to the Mistral url and updates the usage oftool_calls
to match the Mistral API format.