search

microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
22.01k stars 3.28k forks source link

.Net: Bug: Error 405 Method Not Allowed with Mistral Chat Completion #6865

Closed ElieTaillard closed 4 months ago

ElieTaillard commented 5 months ago

Description: I encountered an issue while deploying the Mistral Large model on Azure through an AI hub. When using the method builder.AddMistralChatCompletion(mistralModelName, mistralApiKey, mistralUri), "Mistral") in my kernel builder, everything seems fine. However, when I call the method GetStreamingChatMessageContentsAsync, I receive a 405 Method Not Allowed error.

Reproduction Steps:

  1. Deploy Mistral Large model on Azure AI hub.
  2. Retrieve Mistral API key and URI from the deployment section. image
  3. Use the method builder.AddMistralChatCompletion(mistralModelName, mistralApiKey, mistralUri, "Mistral") with the retrieved API key and URI in the kernel builder.
  4. Call GetStreamingChatMessageContentsAsync in the application.
  5. Execute the application.

Expected Behavior: The GetStreamingChatMessageContentsAsync method should call the correct endpoint and function without any errors.

Actual Behavior: The application calls the endpoint https://Mistral-large-****-serverless.*****.inference.ai.azure.com/chat/completions and receives a 405 Method Not Allowed error.

Workaround: I noticed that only the endpoint with /v1/chat/completions works. To circumvent this issue, I added /v1 to the URI parameter.

Possible Cause: It appears there is an issue with the Mistral connector in the Semantic Kernel. The connector might be missing the /v1 segment in the endpoint path. Additionally, the deployed Mistral model might not accept requests without the /v1 in the URL.

Environment:

markwallace-microsoft commented 5 months ago

@ElieTaillard Our MistralAI connector is intended to be used with the MistralAI platform see: https://docs.mistral.ai/api/.

For Mistral models deployed to Azure we are working on a new connector. This is the tracking task #3992. This is one of our top priorities but we don't have an ETA yet.

ElieTaillard commented 5 months ago

Thank you for the information