vercel / ai

Build AI-powered applications with React, Svelte, Vue, and Solid
https://sdk.vercel.ai/docs
Other
9.36k stars 1.37k forks source link

Unable to Stream #2497

Open KylerD opened 1 month ago

KylerD commented 1 month ago

Description

Version:

"next": "14.2.5"
"ai": "3.2.41"

Using App Router

Route (/api/chat)

  const result = await streamText({
    model: model,
    system: systemContext,
    messages: convertToCoreMessages(messages),
    tools: {
      getKnowledgeSummary: GetKnowledgeSummaryTool,
      checkKnowledge: CheckKnowledgeTool,
    },
  });

  return result.toDataStreamResponse();

Client Component:

 "use client";

import { useChat } from "ai/react";
import { unstable_noStore as noStore } from "next/cache";

export function ChatWindow() {
  noStore();

  const { messages, input, setInput, handleInputChange, handleSubmit, isLoading } = useChat({
    maxToolRoundtrips: 5,
  });

This code streams successfully locally, but not once deployed to vercel. All routes and components in this application are dynamic and follow the suggestions from https://sdk.vercel.ai/docs/troubleshooting/common-issues/streaming-not-working-on-vercel

Any idea what I'm missing?

Code example

No response

Additional context

No response

patrickm68 commented 1 month ago

I updated ai from 2.2.37 to 3.2.37 and now huggingface is not working and is giving 405 error when stream generated from HuggingfaceStream() to StreamTextResponse().

It works locally but fails when deployed on vercel.

https://github.com/vercel/ai/issues/2485