softwaremill / sttp-openai

Apache License 2.0
41 stars 9 forks source link

Added non-openAI backends support #169

Closed lbialy closed 5 months ago

lbialy commented 5 months ago

Closes #166

BaseUri is intentionally made for user to include /v1/, check out the use of openai library in python:

from openai import OpenAI

client = OpenAI(
    base_url = 'http://localhost:11434/v1',
    api_key='ollama', # required, but unused
)

compare:

val client = OpenAISyncClient("ollama", uri"http://localhost:11434/v1")

Things left to figure out:

λ curl https://api.groq.com/openai/v1/chat/completions -w "%{http_code}" \
        -H 'Content-Type: application/json' \
        -d '{
          "model": "mistral",
          "messages": [
            {
              "role": "user",
              "content": "How do I shot web?"
            }
          ]
    }'
{"error":{"message":"Invalid API Key","type":"invalid_request_error","code":"invalid_api_key"}}
401
λ curl https://api.groq.com/openai/v1/chat/completions -w "%{http_code}" \
        -H 'Content-Type: application/json' \
        -H 'Authorization: Bearer REDACTED' \
        -d '{
          "model": "mistral",
          "messages": [
            {
              "role": "user",
              "content": "How do I shot web?"
            }
          ]
    }'
{"error":{"message":"The model `mistral` does not exist or you do not have access to it.","type":"invalid_request_error","code":"model_not_found"}}
404

Now this is probably another issue with upickle because param field is usually defined as param: Option[String] and when it's not present in the json is should be deserialized to None. Maybe we should configure upickle to be more permissive in error json parsing?

lbialy commented 5 months ago

I've fixed missing field error serialization issue in last commit. Hopefully we won't see many other error channel issues with this.

adamw commented 5 months ago

Thanks :)