search

vercel / ai

Build AI-powered applications with React, Svelte, Vue, and Solid
https://sdk.vercel.ai/docs
Other
9.5k stars 1.39k forks source link

Issues generating text with anthropic #3101

Open frmsaul opened 2 hours ago

frmsaul commented 2 hours ago

Description

This works: 

export async function smartPromptRewrite(prompt, suggestions) {
  const stream = createStreamableValue('');

  (async () => {
    const { textStream } = await streamText({
      model: openai('gpt-4o'),
      prompt: promptRewriteMetaprompt(prompt, suggestions),
    });

    for await (const delta of textStream) {
      stream.update(delta);
    }

    stream.done();
  })();

  return { output: stream.value };
}

This doesn't work:

export async function smartPromptRewrite(prompt, suggestions) {
  const stream = createStreamableValue('');

  (async () => {
    const { textStream } = await streamText({
      model: anthropic('claude-3-5-sonnet-20240620'),
      prompt: promptRewriteMetaprompt(prompt, suggestions),
    });

    for await (const delta of textStream) {
      stream.update(delta);
    }

    stream.done();
  })();

  return { output: stream.value };
}

and returns the error: ``The streamable value has been slow to update. This may be a bug or a performance issue or you forgot to call.done()`. Error: Unhandled chunk type: response-metadata at Object.transform (webpack-internal:///(action-browser)/./node_modules/ai/dist/index.mjs:2960:17) at ensureIsPromise (node:internal/webstreams/util:185:19) at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18) at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10) at Object.write (node:internal/webstreams/transformstream:360:14) at ensureIsPromise (node:internal/webstreams/util:185:19) at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5) at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5) at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3) at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3) at [kChunk] (node:internal/webstreams/readablestream:1549:31) at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2091:24) at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2282:5) at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5) at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5) at Object.transform (webpack-internal:///(action-browser)/./node_modules/@ai-sdk/anthropic/dist/index.mjs:525:28) at ensureIsPromise (node:internal/webstreams/util:185:19) at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18) at node:internal/webstreams/transformstream:550:16 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason: Error: Unhandled chunk type: response-metadata at Object.transform (webpack-internal:///(action-browser)/./node_modules/ai/dist/index.mjs:2960:17) at ensureIsPromise (node:internal/webstreams/util:185:19) at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18) at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10) at Object.write (node:internal/webstreams/transformstream:360:14) at ensureIsPromise (node:internal/webstreams/util:185:19) at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5) at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5) at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3) at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3) at [kChunk] (node:internal/webstreams/readablestream:1549:31) at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2091:24) at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2282:5) at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5) at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5) at Object.transform (webpack-internal:///(action-browser)/./node_modules/@ai-sdk/anthropic/dist/index.mjs:525:28) at ensureIsPromise (node:internal/webstreams/util:185:19) at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18) at node:internal/webstreams/transformstream:550:16 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) ⨯ unhandledRejection: Error: Unhandled chunk type: response-metadata at Object.transform (webpack-internal:///(action-browser)/./node_modules/ai/dist/index.mjs:2960:17) at ensureIsPromise (node:internal/webstreams/util:185:19) at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18) at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10) at Object.write (node:internal/webstreams/transformstream:360:14) at ensureIsPromise (node:internal/webstreams/util:185:19) at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5) at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5) at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3) at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3) at [kChunk] (node:internal/webstreams/readablestream:1549:31) at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2091:24) at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2282:5) at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5) at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5) at Object.transform (webpack-internal:///(action-browser)/./node_modules/@ai-sdk/anthropic/dist/index.mjs:525:28) at ensureIsPromise (node:internal/webstreams/util:185:19) at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18) at node:internal/webstreams/transformstream:550:16 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) ⨯ unhandledRejection: Error: Unhandled chunk type: response-metadata at Object.transform (webpack-internal:///(action-browser)/./node_modules/ai/dist/index.mjs:2960:17) at ensureIsPromise (node:internal/webstreams/util:185:19) at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18) at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10) at Object.write (node:internal/webstreams/transformstream:360:14) at ensureIsPromise (node:internal/webstreams/util:185:19) at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5) at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5) at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3) at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3) at [kChunk] (node:internal/webstreams/readablestream:1549:31) at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2091:24) at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2282:5) at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5) at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5) at Object.transform (webpack-internal:///(action-browser)/./node_modules/@ai-sdk/anthropic/dist/index.mjs:525:28) at ensureIsPromise (node:internal/webstreams/util:185:19) at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18) at node:internal/webstreams/transformstream:550:16 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)


FWIW, this works:

console.log('regenerating Prompt:', prompt); let text = await generateText({ model: anthropic('claude-3-5-sonnet-20240620'), prompt: promptRewriteMetaprompt(prompt, suggestions), }); console.log('text', text);


### Additional context

versions: 

"@ai-sdk/anthropic": "^0.0.50",
"@ai-sdk/openai": "^0.0.50",
 "ai": "^3.3.12",
lgrammel commented 2 hours ago

Can you try this example:

https://github.com/vercel/ai/blob/main/examples/ai-core/src/stream-text/anthropic.ts

(it works for me)

frmsaul commented 1 hour ago

Hi @lgrammel,

Thanks for reaching out. I tried running your script. This is the error that i'm getting: 

Error: Unhandled chunk type: response-metadata
    at Object.transform (file:///Users/saulfuhrmann/dev/breezy-fe/node_modules/ai/dist/index.mjs:2894:17)
    at ensureIsPromise (node:internal/webstreams/util:185:19)
    at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18)
    at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10)
    at Object.write (node:internal/webstreams/transformstream:360:14)
    at ensureIsPromise (node:internal/webstreams/util:185:19)
    at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5)
    at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5)
    at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3)
    at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3)
lgrammel commented 1 hour ago

@frmsaul can you update @ai-sdk/anthropic to 0.0.50 and ai to 3.4.1 ? Most likely your ai sdk version is too old for the provider