feat: Images support to `VercelUseChatRuntime`

[Rajaniraiyn]

images content type for user message when using useVercelChatRuntime

import { useAppendMessage } from "@assistant-ui/react";

const CustomButton = () => {
  const append = useAppendMessage();

  const handleClick = () => {
    append({
    role: "user",
    content: [{type: "image", image: "https://example.com/example.png"}]
    });
  };

  return <button onClick={handleClick}>Send</button>;
};

[vercel[bot]]

@Rajaniraiyn is attempting to deploy a commit to the Simon Farshid's projects Team on Vercel.

A member of the Team first needs to authorize it.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
assistant-ui	❌ Failed (Inspect)			Jul 28, 2024 4:01pm

[Yonom]

Thank you so much for this! I really appreciate it.

I would love to add support for images, however the Vercel useChat runtime doesn't seem to natively support appending images yet. The typescript types state that Message.content is a string. Your method only accidentally works if the backend endpoint does not use "convertToCoreMessages(messages)" if the user adds experimental_attachments, the image functionality breaks

I'm afraid I can't accept this until Vercel AI SDK adds native image support on their side - I don't want to diverge from their spec here.

The Edge runtime supports images in the meantime and is based on the AI SDK

[Rajaniraiyn]

Thanks for the feedback,

Totally get it. Maybe the error message could mention the Edge runtime for more clarity.

[Yonom]

@Rajaniraiyn good idea - done https://github.com/Yonom/assistant-ui/pull/602