Zod validation failing specifically for "cognitivecomputations/dolphin-mixtral-8x22b"

[ichabodcole]

Hi, I'm currently using the vercel sdk using the steamText function, and currently hitting an issue with the response from a specific model cognitivecomputations/dolphin-mixtral-8x22b, other models I have tested seem to work as expected. I'm not sure if the issue is with this library or the Vercel library, but I figured I'd let check in here first.

Below is the validation error I'm receiving.

"Type validation failed: Value: {"id":"gen-9KTZBnGp8T1l51fYzRMuvaAHwbtt","provider":"Novita","model":"cognitivecomputations/dolphin-mixtral-8x22b","object":"chat.completion.chunk","created":1727133797,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null,"logprobs":{"tokens":null,"token_logprobs":null,"top_logprobs":null,"text_offset":null}}]}.
Error message: [
  {
    "code": "invalid_union",
    "unionErrors": [
      {
        "issues": [
          {
            "code": "invalid_type",
            "expected": "array",
            "received": "undefined",
            "path": [
              "choices",
              0,
              "logprobs",
              "content"
            ],
            "message": "Required"
          }
        ],
        "name": "ZodError"
      },
      {
        "issues": [
          {
            "code": "invalid_type",
            "expected": "object",
            "received": "undefined",
            "path": [
              "error"
            ],
            "message": "Required"
          }
        ],
        "name": "ZodError"
      }
    ],
    "path": [],
    "message": "Invalid input"
  }
]"

Here is the streaming response data which causes the Zod validation error, it appears to be due to the "logprobs" value.

{
  "id": "gen-9KTZBnGp8T1l51fYzRMuvaAHwbtt",
  "provider": "Novita",
  "model": "cognitivecomputations/dolphin-mixtral-8x22b",
  "object": "chat.completion.chunk",
  "created": 1727133797,
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": "he "
      },
      "finish_reason": null,
      "logprobs": {
        "tokens": null,
        "token_logprobs": null,
        "top_logprobs": null,
        "text_offset": null
      }
    }
  ]
}

However the below succeeds with no issues, with logprobs set to null.

{
  "id": "gen-YMizbTfPRjAeRt85myCTbAadX5tg",
  "provider": "Lepton",
  "model": "cognitivecomputations/dolphin-mixtral-8x7b",
  "object": "chat.completion.chunk",
  "created": 1727134025,
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": "iolets are"
      },
      "finish_reason": null,
      "logprobs": null
    }
  ]
}

[ichabodcole]

Issue appears to be with map-openrouter-chat-logprobs.ts.

export function mapOpenRouterChatLogProbsOutput(
  logprobs: OpenRouterChatLogProbs | null | undefined
): LanguageModelV1LogProbs | null | undefined {
  return (
    logprobs?.content?.map(({ token, logprob, top_logprobs }) => ({
      token,
      logprob,
      topLogprobs: top_logprobs
        ? top_logprobs.map(({ token, logprob }) => ({
            token,
            logprob,
          }))
        : [],
    }))
  );
}