streamText with image data for Anthropic on Edge Runtime fires Illegal invocation error

[twxxk]

Description

1. Call experimental_streamText with text messages for an anthropic model. It works as expected.
2. Add the base64 image and ask the model to describe the image.

It works on the local backend environment, but when the edge runtime triggers the following error.

undefined 'TypeError: Illegal invocation' TypeError: Illegal invocation at (node_modules/.pnpm/ai@3.0.21_react@18.2.0_solid-js@1.8.16_svelte@4.2.13_vue@3.4.21_zod@3.22.4/node_modules/ai/dist/index.mjs:78:34) at (node_modules/.pnpm/ai@3.0.21_react@18.2.0_solid-js@1.8.16_svelte@4.2.13_vue@3.4.21_zod@3.22.4/node_modules/ai/dist/index.mjs:130:37) at (node_modules/.pnpm/ai@3.0.21_react@18.2.0_solid-js@1.8.16_svelte@4.2.13_vue@3.4.21_zod@3.22.4/node_modules/ai/dist/index.mjs:181:26) at (node_modules/.pnpm/ai@3.0.21_react@18.2.0_solid-js@1.8.16_svelte@4.2.13_vue@3.4.21_zod@3.22.4/node_modules/ai/dist/index.mjs:165:41) at (node_modules/.pnpm/ai@3.0.21_react@18.2.0_solid-js@1.8.16_svelte@4.2.13_vue@3.4.21_zod@3.22.4/node_modules/ai/dist/index.mjs:154:27) at (node_modules/.pnpm/ai@3.0.21_react@18.2.0_solid-js@1.8.16_svelte@4.2.13_vue@3.4.21_zod@3.22.4/node_modules/ai/dist/index.mjs:1429:43) at (node_modules/.pnpm/ai@3.0.21_react@18.2.0_solid-js@1.8.16_svelte@4.2.13_vue@3.4.21_zod@3.22.4/node_modules/ai/dist/index.mjs:405:17) at (node_modules/.pnpm/ai@3.0.21_react@18.2.0_solid-js@1.8.16_svelte@4.2.13_vue@3.4.21_zod@3.22.4/node_modules/ai/dist/index.mjs:394:24)

Reproduced with 3.0.21

Code example

export const runtime = 'edge';

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY})
const messages = [
  { type: "text", text: message },
  { type: "image", image: image_data, mimeType: image_media_type }
];
const result = await experimental_streamText({
    model: anthropic.messages('claude-3-sonnet-20240229'),
    messages: messages as ExperimentalMessage[],
});
const stream = result.toAIStream();
return new StreamingTextResponse(stream);

Additional context

It seems the image data is being converted to UInt8Array but globalThis.atob(base64Url) does not work on the edge runtime. https://github.com/vercel/ai/blob/51f25a7f7e7b10ab2d46c96ea36220f927adf6b4/packages/core/core/prompt/convert-to-language-model-prompt.ts#L61 https://github.com/vercel/ai/blob/51f25a7f7e7b10ab2d46c96ea36220f927adf6b4/packages/core/core/prompt/data-content.ts#L36 https://github.com/vercel/ai/blob/3fd1918c4e4a06929b4e8f4b65c4676a02151cb5/packages/provider-utils/src/uint8-utils.ts#L1

[lgrammel]

globalThis.atob should be available on the edge runtime: https://vercel.com/docs/functions/runtimes/edge-runtime

@twxxk how are you invoking the code example? Inside a route for the app router?

[ttanida]

Hi @twxxk, I am facing the exact same issue.

When I try to send an image using streamText (ai version 3.1.8) in my app router, it works in the local and non-edge environments, but not with edge runtime.

I convert my image to Uint8Array (which seems to be what the ai source code does in the background) before appending it to the messages array. However, in my deployment logs, I get:

[POST] /api/chat reason=EDGE_FUNCTION_INVOCATION_FAILED, status=500, user_error=true Status Code 405 Method Not Allowed

Would you mind sharing any tips on solving this issue? I would appreciate it!

[lgrammel]

@ttanida can you share you client & server code?

[ttanida]

Hi @lgrammel,

thanks for taking a look at this! I used:

ai@3.1.12 ai-sdk/openai@0.0.11

On the client-side, I take a screenshot of my screen using:

canvasElement.width = videoElement.videoWidth;
canvasElement.height = videoElement.videoHeight;
canvasContext.drawImage(
      videoElement,
      0,
      0,
      canvasElement.width,
      canvasElement.height
    );

const imageUrl = canvasElement.toDataURL("image/jpeg", 1); // Quality range is 0 to 1

The screen shot imageUrl is then sent to my chat API inside the messages array. On server-side, my app/api/chat/route.js looks like this:

import { openai } from "@ai-sdk/openai";
import { StreamingTextResponse, streamText } from "ai";

export const runtime = "edge";
const model = "gpt-4o";

function dataURLToUint8Array(dataURL) {
  const base64String = dataURL.split(",")[1]; // Remove the Data URL prefix
  const buffer = Buffer.from(base64String, "base64");
  return new Uint8Array(buffer);
}

export async function POST(req) {
  const { messages } = await req.json();

  let lastElement = messages[messages.length - 1];
  if (Array.isArray(lastElement.content)) {
    const imageUrl = lastElement.content[1].image;
    const firstPart = imageUrl.slice(0, 40);
    const lastPart = imageUrl.slice(-40);
    console.log(`${firstPart}...${lastPart}`);

    // Convert the Data URL to Uint8Array
    lastElement.content[1].image = dataURLToUint8Array(
      lastElement.content[1].image
    );
  }

  const result = await streamText({
    model: openai(model),
    system: "You are a helpful assistant.",
    messages,
  });

  return new StreamingTextResponse(result.toAIStream());
}

The logged imageUrl looks like this:

data:image/jpeg;base64,/9j/4AAQSkZJRgABA...Jh6tUF9JricpNomQZsLLz+Lw1GXezv4bj8z6r//Z

The messages array looks like this:

[
  {
  role: 'assistant',
  content: 'Hello Tim! How can I help you today?'
},
  {
  role: 'user',
  content: [
  { type: 'text', text: 'Tell me what you see in the image.’ },
  { type: 'image', image: theImage }
]
}
]

where theImage is the output of dataURLToUint8Array.

The chat works perfectly (both with and without images present in the messages array) when executed locally or non-edge run time (i.e. commenting out line "export const runtime = "edge";").

The chat works perfectly in edge run time only when no images are present in messages. When an image is present (like the above example), then I get the warning:

[POST] /api/chat reason=EDGE_FUNCTION_INVOCATION_FAILED, status=500, user_error=true

Status Code
405
Method Not Allowed

and the error:

Error:

TypeError: Right-hand side of 'instanceof' is not an object
    at (node_modules/.pnpm/ai@3.1.12_openai@4.47.1_react@18.2.0_solid-js@1.8.17_svelte@4.2.17_vue@3.4.27_zod@3.23.8/node_modules/ai/dist/index.mjs:34:20)

However, the error seems to me more like the symptome of the underlying issue and not the cause?

I would appreciate if you could help me solve this issue!

[lucasishuman]

Having same issue - sending image data to OpenAI gpt-4-turbo works locally, but not on Vercel using Edge.

Sending messages without image data or an image URL streams fine on Vercel without any other changes.

"ai": "3.1.12", "@ai-sdk/anthropic": "0.0.14", "@ai-sdk/google": "0.0.14", "@ai-sdk/openai": "0.0.13",

Seeing in Vercel logs:

[POST] /api/chat reason=EDGE_FUNCTION_INVOCATION_FAILED, status=500, user_error=true

TypeError: Right-hand side of 'instanceof' is not an object
    at (../../node_modules/@ai-sdk/provider-utils/dist/index.mjs:33:0)
    at (../../node_modules/ai/dist/index.mjs:34:20)
    at (../../node_modules/ai/dist/index.mjs:21:6)
    at (../../node_modules/ai/dist/index.mjs:1456:42)
    at (src/app/api/chat/route.ts:121:34)
    at (../../node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:207:0)
    at (../../node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:122:0)
    at (../../node_modules/next/dist/esm/server/future/route-modules/app-route/module.js:269:0)
    at (../../node_modules/next/dist/esm/server/web/edge-route-module-wrapper.js:81:0)
    at (../../node_modules/next/dist/esm/server/web/adapter.js?4b18:158:0)

[twxxk]

Yes, I also get 'instanceof' error with 3.1.12. Workaround: If you disable runtime = 'edge', the issue does not happen as ttanida said.

I would also mention the official document says the function accepts the image parameter as ArrayBuffer | Uint8Array | Buffer | URL but passing the URL to Anthropic throws "URL image parts' functionality not supported."

[ttanida]

Hello @lgrammel,

are there any updates on this (maybe the issue should be reopened)?

To make processing image data on the edge possible, I am currently using a previous version of ai:

ai@3.0.31 ai-sdk/openai@0.0.11

However, if possible, I would like to update to the latest version.

[lgrammel]

There are several issues going on here.

@twxxk Anthropic does not support URL image parts. https://github.com/vercel/ai/issues/1817

@lucasishuman I've traced this to

// src/is-abort-error.ts
function isAbortError(error) {
  return error instanceof DOMException && (error.name === "AbortError" || error.name === "TimeoutError"); // line 33
}

my hunch is that DOMException is not available as an object on Edge. The Edge docs say it's available tho.

[lucasishuman]

Thanks for the additional info - I disabled 'edge' and that unblocked me for now.

[sarnakov]

Got the same TypeError using OpenAI for only text generation. That can be fixed by using node.js runtime instead of edge. Though, I'd appreciate if it could be fixed to use edge runtime.