OpenAI compatible API provider does not work (FriendliAI)

[minpeter]

Description

referring to providers/ai-sdk-providers/fireworks and docs/getting-started/nextjs-app-router, and got the following results.

    presence_penalty: undefined,
    seed: undefined,
    messages: [ [Object] ],
    tools: undefined,
    tool_choice: undefined,
    stream: true,
    stream_options: undefined
  },
  statusCode: 422,
  responseHeaders: {
    'access-control-allow-credentials': 'true',
    connection: 'keep-alive',
    'content-length': '107',
    'content-type': 'application/json',
    date: 'Thu, 30 May 2024 11:37:11 GMT',
    server: 'Orca',
    vary: 'Origin',
    'x-ratelimit-limit-requests': '10',
    'x-ratelimit-limit-tokens': '10000',
    'x-ratelimit-remaining-requests': '9',
    'x-ratelimit-remaining-tokens': '10000',
    'x-ratelimit-reset-requests': '60',
    'x-ratelimit-reset-tokens': '60'
  },
  responseBody: '{"message":"invalid JSON payload: (messages[0]) content: Proto field is not repeating, cannot start list."}',
  cause: undefined,
  isRetryable: false,
  data: undefined
}
 POST /api/chat 500 in 384ms

In the latter case, it returns that the content of the messages field cannot begin with a list.

I think this is a problem with the recently changed role of openai's messages field being changed to the form below instead of a string.

"messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "What'\''s in this image?"
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
            }
          }
        ]
      }
    ],

This issue has not been reported yet, but I believe it is a possible issue for most providers that use the createOpenAI method of@ai-sdk/openai`. (Perplexity - Perplexity api docs)

To solve this problem, I would like to suggest changing the user role's message to also be sent as a string once the compatibility option in createOpenAI is set to compatibility: "compatible" .

*I like the code that I use for the added features. (ref: https://github.com/vercel/ai/commit/2b18fa11ba343bf09eb879607da41fc3f4fa7599) I don't think it's possible to use this code, but it's recommended to use the API provider. https://github.com/vercel/ai/blob/8e1f7e395daa8e75abcb33c180f9dd0f59cba570/packages/openai/src/openai-chat-language-model.ts#L406

| Lastly, I apologize for my poor English and explanation.

Code example

app/api/chat/route.ts

import { streamText } from "ai";

import { createOpenAI } from "@ai-sdk/openai";

// const fireworks = createOpenAI({
//   apiKey: process.env.FIREWORKS_API_KEY ?? "",
//   baseURL: "https://api.fireworks.ai/inference/v1",
//   compatibility: "compatible",
// });

const friendliai = createOpenAI({
  apiKey: process.env.FRIENDLI_API_KEY ?? "",
  baseURL: "https://inference.friendli.ai/v1",
  compatibility: "compatible",
});
export async function POST(req: Request) {
  const { messages } = await req.json();

  // const result = await streamText({
  //   model: fireworks("accounts/fireworks/models/mixtral-8x7b-instruct"),
  //   messages,
  // });

  const result = await streamText({
    model: friendliai("mixtral-8x7b-instruct-v0-1"),
    messages,
  });

  return result.toAIStreamResponse();
}

Additional context

Completions that send the messages[0].content field entirely as a string succeed successfully on all providers. However, in this case, function calling is not possible.

[minpeter]

Also, after applying the PR below, the example of fireworks AI, which was not working for some reason, started to stop working.

I can't test it with the subtitle of free credits, but it's probably a positive patch for groq and Perplexity as well.

[lgrammel]

https://github.com/vercel/ai/pull/1765