`createAI` and `render` function from `ai/rsc` returns `any`

[dennis-adamczyk]

Description

I tried to experiment with Generative UI. Therefore, I followed the steps described on https://sdk.vercel.ai/docs/concepts/ai-rsc#setup. After copying and pasting the code in step 2 I get a few type errors. The const AI has the type of any when hovering over it in VSCode. Also, the ui const which is the result of calling render throws an error, saying it is typed as any. But when hovering over the functions themselves, I see that they have a correct return type. You can check out the code at https://github.com/dennis-adamczyk/generative-ui-example but I really did just follow the steps in the documentation.

Code example

// app/action.tsx
// see https://sdk.vercel.ai/docs/concepts/ai-rsc#create-an-airsc-instance-on-the-server
import { OpenAI } from 'openai';
import { createAI, getMutableAIState, render } from 'ai/rsc';
import { z } from 'zod';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

// An example of a spinner component. You can also import your own components,
// or 3rd party component libraries.
function Spinner() {
  return <div>Loading...</div>;
}

// An example of a flight card component.
function FlightCard({ flightInfo }) { // -> type error: Binding element 'flightInfo' implicitly has an 'any' type. ts(7031)
  return (
    <div>
      <h2>Flight Information</h2>
      <p>Flight Number: {flightInfo.flightNumber}</p>
      <p>Departure: {flightInfo.departure}</p>
      <p>Arrival: {flightInfo.arrival}</p>
    </div>
  );
}

// An example of a function that fetches flight information from an external API.
async function getFlightInfo(flightNumber: string) {
  return {
    flightNumber,
    departure: 'New York',
    arrival: 'San Francisco',
  };
}

async function submitUserMessage(userInput: string) {
  'use server';

  const aiState = getMutableAIState<typeof AI>();
  // -> type error: 'aiState' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly in its own initializer. ts(7022)

  // Update the AI state with the new user message.
  aiState.update([
    ...aiState.get(),
    {
      role: 'user',
      content: userInput,
    },
  ]);

  // The `render()` creates a generated, streamable UI.
  const ui = render({ // -> type error: 'ui' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly in its own initializer. ts(7022)
    model: 'gpt-4-0125-preview',
    provider: openai,
    messages: [
      { role: 'system', content: 'You are a flight assistant' },
      ...aiState.get(),
    ],
    // `text` is called when an AI returns a text response (as opposed to a tool call).
    // Its content is streamed from the LLM, so this function will be called
    // multiple times with `content` being incremental.
    text: ({ content, done }) => {
      // When it's the final content, mark the state as done and ready for the client to access.
      if (done) {
        aiState.done([
          ...aiState.get(),
          {
            role: 'assistant',
            content,
          },
        ]);
      }

      return <p>{content}</p>;
    },
    tools: {
      get_flight_info: {
        description: 'Get the information for a flight',
        parameters: z
          .object({
            flightNumber: z.string().describe('the number of the flight'),
          })
          .required(),
        render: async function* ({ flightNumber }) {
          // Show a spinner on the client while we wait for the response.
          yield <Spinner />;

          // Fetch the flight information from an external API.
          const flightInfo = await getFlightInfo(flightNumber);

          // Update the final AI state.
          aiState.done([
            ...aiState.get(),
            {
              role: 'function',
              name: 'get_flight_info',
              // Content can be any string to provide context to the LLM in the rest of the conversation.
              content: JSON.stringify(flightInfo),
            },
          ]);

          // Return the flight card to the client.
          return <FlightCard flightInfo={flightInfo} />;
        },
      },
    },
  });

  return {
    id: Date.now(),
    display: ui,
  };
}

// Define the initial state of the AI. It can be any JSON object.
const initialAIState: {
  role: 'user' | 'assistant' | 'system' | 'function';
  content: string;
  id?: string;
  name?: string;
}[] = [];

// The initial UI state that the client will keep track of, which contains the message IDs and their UI nodes.
const initialUIState: {
  id: number;
  display: React.ReactNode;
}[] = [];

// AI is a provider you wrap your application with so you can access AI and UI state in your components.
export const AI = createAI({ // -> type error: 'AI' implicitly has type 'any' because it does not have a type annotation and is referenced directly or indirectly in its own initializer. ts(7022)
  actions: {
    submitUserMessage,
  },
  // Each state can be any shape of object, but for chat applications
  // it makes sense to have an array of messages. Or you may prefer something like { id: number, messages: Message[] }
  initialUIState,
  initialAIState,
});

Additional context

I'm running node v20.9.0 and TypeScript v5.3.2 on Windows 11 (WSL).

[unstubbable]

If I remember correctly, the type of AI cannot be inferred correctly with the implicit return type of the action. For me, it helped to set an explicit return type, see https://github.com/unstubbable/ai-rsc-test/commit/ef44eb0.

[dennis-adamczyk]

That's correct. Because there is a cycling dependency in the definition of the AIProvider, the type can not be inferred. This is fixable when explicitly type the return type of the action (like here).

However, there is still a type error in the submitUserMessage function, because aiState.get() is not compatible with the messages property of the render function: see this line. The error is the following:

Type '{ role: "function" | "user" | "assistant" | "system"; content: string; id?: string | undefined; name?: string | undefined; }' is not assignable to type 'ChatCompletionMessageParam'.
  Type '{ role: "function" | "user" | "assistant" | "system"; content: string; id?: string | undefined; name?: string | undefined; }' is not assignable to type 'ChatCompletionFunctionMessageParam'.
    Types of property 'name' are incompatible.
      Type 'string | undefined' is not assignable to type 'string'.
        Type 'undefined' is not assignable to type 'string'. ts(2322)

After inspecting the types I still do not have a clue why the name property should not be compatible. I guess, I need some help here.

[unstubbable]

Define a proper union type to fix this: https://github.com/unstubbable/ai-rsc-test/blob/ef44eb040b8afaee86a6b7c3665f19983748998d/app/action.tsx#L15

[parthematics]

@unstubbable @dennis-adamczyk I have a question slightly related to this but how do you specify a maxDuration for the async render functions within the tool calls? e.g. render: async function* ({ flightNumber }) ...

I noticed that upgrading to the pro plan increases the function time limit from 10s to 15s, but how can I specify these on the fly here (or define a global default)? Since the functions are being invoked from an actions.tsx file as opposed to the standard app/api/**/* pattern, I'm getting errors when trying to define these globally in a vercel.json file. Any help would be appreciated here, and I'm happy to open up a new issue for this.

[MaxLeiter]

@parthematics actions inherit runtime configurations from the page/layout they're called from (this is because they're ultimately in the same bundle on the server). See https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config

[parthematics]

@parthematics actions inherit runtime configurations from the page/layout they're called from (this is because they're ultimately in the same bundle on the server). See https://nextjs.org/docs/app/api-reference/file-conventions/route-segment-config

Ah, gotcha. I'll try that and see if that resolves it! Thanks :)