vercel / ai-chatbot

A full-featured, hackable Next.js AI chatbot built by Vercel
https://chat.vercel.ai
Other
6.04k stars 1.85k forks source link

Stalling on some threads randomly - `undefined` runResult for certain threads. #381

Open jai2201 opened 2 months ago

jai2201 commented 2 months ago

I'm experiencing this weird issue of getting undefined runResult for certain threads and it is very random, I am using useAssistant hook and it does work fine for 80% of the cases, but some times i am just getting undefined in runResult even when all of the other things have been created properly - (thread, message, assistant, etc)

code ->

    const createdMessage = await openai.beta.threads.messages.create(
      threadId,
      {
        role: 'user',
        content: reqBody.message,
      },
      { signal: req.signal }
    );

    const systemMessage = await getAssistantSystemMessage(
      supabase,
      user?.user?.id
    );

    return AssistantResponse(
      { threadId, messageId: createdMessage.id },
      async ({ forwardStream }) => {
        // Run the assistant on the thread
        const runStream = openai.beta.threads.runs.stream(
          threadId,
          {
            assistant_id:
              reqBody.data.assistantId ??
              (() => {
                throw new Error('ASSISTANT_ID is not set');
              })(),
            instructions: systemMessage,
          },
          { signal: req.signal }
        );
        // forward run status would stream message deltas
        let runResult = await forwardStream(runStream);
        console.log(runResult, 'runResult');

        // status can be: queued, in_progress, requires_action, cancelling, cancelled, failed, completed, or expired
        while (
          runResult?.status === 'requires_action' &&
          runResult.required_action?.type === 'submit_tool_outputs'
        ) {
          const toolCalls =
            runResult.required_action.submit_tool_outputs.tool_calls;
          const toolOutputs = [];

          for (const toolCall of toolCalls) {
            // Extract the function name from the tool call
            const functionName = toolCall.function.name;

            // Parse the function arguments from the tool call
            const functionArgs = JSON.parse(toolCall.function.arguments);
            const assistantFunctionsMap: Record<string, Function> = {
              searchPersonByTitleKeywords: async (
                args: any,
                userId: string,
                supabase: SupabaseClient
              ) =>
                searchPersonByTitleKeywords(
                  args.titleKeywords,
                  userId,
                  supabase
                ),
              searchPersonByName: async (
                args: any,
                userId: string,
                supabase: SupabaseClient
              ) => searchPersonByName(args.name, userId, supabase),
              getPersonDetails: async (
                args: any,
                userId: string,
                supabase: SupabaseClient
              ) => getFormattedPersonString(args.personId, userId, supabase),
            };
            // Look up the actual function to call based on the function name
            const functionToCall = assistantFunctionsMap[functionName];
            const functionResponse = await functionToCall(
              functionArgs,
              user?.user.id,
              supabase
            );

            const outputString = JSON.stringify(functionResponse);

            toolOutputs.push({
              tool_call_id: toolCall.id,
              output: outputString,
            });
          }

          runResult = await forwardStream(
            openai.beta.threads.runs.submitToolOutputsStream(
              threadId,
              runResult.id,
              { tool_outputs: toolOutputs },
              { signal: req.signal }
            )
          );
        }
      }
    );
  }
jai2201 commented 2 months ago

when i do console.log(runResult) for those particular threads, it returns undefined to me, there's no error which is thrown out, nothing else happens, just no response.

jai2201 commented 2 months ago

Here's the picture ->

image

Now this same prompt if i give to some new thread, it'll return me a response, it's just that for this particular thread, i won't get a response at all.