search

vercel / ai

Build AI-powered applications with React, Svelte, Vue, and Solid
https://sdk.vercel.ai/docs
Other
9.58k stars 1.41k forks source link

[perplexity] streamObject fails #2061

Closed louis030195 closed 1 month ago

louis030195 commented 3 months ago

Description

// app/api/cook/route.ts
import { createOpenAI } from "@ai-sdk/openai";
import { streamObject } from "ai";
import { NextResponse } from "next/server";
import { z } from "zod";

if (!process.env.PERPLEXITY_API_KEY) {
  throw new Error("PERPLEXITY_API_KEY is not set");
}

export async function POST(request: Request): Promise<NextResponse> {
  const body = await request.json();
  const perplexity = createOpenAI({
    apiKey: process.env.PERPLEXITY_API_KEY!,
    baseURL: "https://api.perplexity.ai/",
  });
  const model = perplexity("llama-3-sonar-large-32k-online");

  const result = await streamObject({
    schema: z.object({
      recipes: z.array(
        z.object({
          name: z.string(),
          ingredients: z.array(z.string()),
          steps: z.array(z.string()),
          url: z.string(),
        })
      ),
    }),
    model: model,
    messages: [
      {
        role: "system",
        content:
          "You are an AI assistant that helps people find the best ways to cook things." +
          "Make sure to mention the URL of the sources of your answers.",
      },
      {
        role: "user",
        content: `Top 10 best ways to cook "${body.prompt}"`,
      },
    ],
  });

  // use textStream as an async iterable:
  for await (const textPart of result.textStream) {
    console.log(textPart);
  }
  try {
    return NextResponse.json({});
  } catch (error) {
    return NextResponse.json(
      { error: (error as Error).message },
      { status: 400 }
    );
  }
}

// curl -X POST -H "Content-Type: application/json" -d '{"prompt": "broccoli w tofu vegan w spirulina"}' http://localhost:3000/api/cook

Code example

No response

Additional context

TypeValidationError [AI_TypeValidationError]: Type validation failed: Value: undefined.
Error message: [
  {
    "code": "invalid_type",
    "expected": "object",
    "received": "undefined",
    "path": [],
    "message": "Required"
  }
]
    at safeValidateTypes (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:224:14)
    at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:986:113)
    ... 42 lines matching cause stack trace ...
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  cause: ZodError: [
    {
      "code": "invalid_type",
      "expected": "object",
      "received": "undefined",
      "path": [],
      "message": "Required"
    }
  ]
      at get error [as error] (webpack-internal:///(rsc)/./node_modules/.pnpm/zod@3.23.8/node_modules/zod/lib/index.mjs:699:31)
      at safeValidateTypes (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:226:33)
      at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:986:113)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10)
      at Object.write (node:internal/webstreams/transformstream:360:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3)
      at [kChunk] (node:internal/webstreams/readablestream:1545:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2087:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2278:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5)
      at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:902:26)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10)
      at Object.write (node:internal/webstreams/transformstream:360:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3)
      at [kChunk] (node:internal/webstreams/readablestream:1545:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2087:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2278:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5)
      at Object.flush (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+openai@0.0.33_zod@3.23.8/node_modules/@ai-sdk/openai/dist/index.mjs:418:24)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultSinkCloseAlgorithm (node:internal/webstreams/transformstream:569:5)
      at Object.close (node:internal/webstreams/transformstream:366:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessClose (node:internal/webstreams/writablestream:1142:28)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1222:5)
      at writableStreamDefaultControllerClose (node:internal/webstreams/writablestream:1189:3)
      at writableStreamClose (node:internal/webstreams/writablestream:699:3)
      at writableStreamDefaultWriterClose (node:internal/webstreams/writablestream:1071:10)
      at writableStreamDefaultWriterCloseWithErrorPropagation (node:internal/webstreams/writablestream:1063:10)
      at node:internal/webstreams/readablestream:1517:15
      at complete (node:internal/webstreams/readablestream:1396:9)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
    issues: [ [Object] ],
    addIssue: [Function (anonymous)],
    addIssues: [Function (anonymous)],
    errors: [ [Object] ]
  },
  value: undefined
}
 ⨯ unhandledRejection: TypeValidationError [AI_TypeValidationError]: Type validation failed: Value: undefined.
Error message: [
  {
    "code": "invalid_type",
    "expected": "object",
    "received": "undefined",
    "path": [],
    "message": "Required"
  }
]
    at safeValidateTypes (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:224:14)
    at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:986:113)
    ... 42 lines matching cause stack trace ...
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  cause: ZodError: [
    {
      "code": "invalid_type",
      "expected": "object",
      "received": "undefined",
      "path": [],
      "message": "Required"
    }
  ]
      at get error [as error] (webpack-internal:///(rsc)/./node_modules/.pnpm/zod@3.23.8/node_modules/zod/lib/index.mjs:699:31)
      at safeValidateTypes (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:226:33)
      at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:986:113)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10)
      at Object.write (node:internal/webstreams/transformstream:360:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3)
      at [kChunk] (node:internal/webstreams/readablestream:1545:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2087:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2278:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5)
      at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:902:26)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10)
      at Object.write (node:internal/webstreams/transformstream:360:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3)
      at [kChunk] (node:internal/webstreams/readablestream:1545:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2087:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2278:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5)
      at Object.flush (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+openai@0.0.33_zod@3.23.8/node_modules/@ai-sdk/openai/dist/index.mjs:418:24)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultSinkCloseAlgorithm (node:internal/webstreams/transformstream:569:5)
      at Object.close (node:internal/webstreams/transformstream:366:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessClose (node:internal/webstreams/writablestream:1142:28)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1222:5)
      at writableStreamDefaultControllerClose (node:internal/webstreams/writablestream:1189:3)
      at writableStreamClose (node:internal/webstreams/writablestream:699:3)
      at writableStreamDefaultWriterClose (node:internal/webstreams/writablestream:1071:10)
      at writableStreamDefaultWriterCloseWithErrorPropagation (node:internal/webstreams/writablestream:1063:10)
      at node:internal/webstreams/readablestream:1517:15
      at complete (node:internal/webstreams/readablestream:1396:9)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
    issues: [ [Object] ],
    addIssue: [Function (anonymous)],
    addIssues: [Function (anonymous)],
    errors: [ [Object] ]
  },
  value: undefined
}
 ⨯ unhandledRejection: TypeValidationError [AI_TypeValidationError]: Type validation failed: Value: undefined.
Error message: [
  {
    "code": "invalid_type",
    "expected": "object",
    "received": "undefined",
    "path": [],
    "message": "Required"
  }
]
    at safeValidateTypes (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:224:14)
    at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:986:113)
    ... 42 lines matching cause stack trace ...
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  cause: ZodError: [
    {
      "code": "invalid_type",
      "expected": "object",
      "received": "undefined",
      "path": [],
      "message": "Required"
    }
  ]
      at get error [as error] (webpack-internal:///(rsc)/./node_modules/.pnpm/zod@3.23.8/node_modules/zod/lib/index.mjs:699:31)
      at safeValidateTypes (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+provider-utils@0.0.16_zod@3.23.8/node_modules/@ai-sdk/provider-utils/dist/index.mjs:226:33)
      at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:986:113)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10)
      at Object.write (node:internal/webstreams/transformstream:360:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3)
      at [kChunk] (node:internal/webstreams/readablestream:1545:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2087:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2278:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5)
      at Object.transform (webpack-internal:///(rsc)/./node_modules/.pnpm/ai@3.2.5_openai@4.52.0_react@18.3.1_solid-js@1.8.17_svelte@4.2.18_vue@3.4.29_zod@3.23.8/node_modules/ai/dist/index.mjs:902:26)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultControllerPerformTransform (node:internal/webstreams/transformstream:505:18)
      at transformStreamDefaultSinkWriteAlgorithm (node:internal/webstreams/transformstream:555:10)
      at Object.write (node:internal/webstreams/transformstream:360:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessWrite (node:internal/webstreams/writablestream:1109:5)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1224:5)
      at writableStreamDefaultControllerWrite (node:internal/webstreams/writablestream:1098:3)
      at writableStreamDefaultWriterWrite (node:internal/webstreams/writablestream:988:3)
      at [kChunk] (node:internal/webstreams/readablestream:1545:31)
      at readableStreamFulfillReadRequest (node:internal/webstreams/readablestream:2087:24)
      at readableStreamDefaultControllerEnqueue (node:internal/webstreams/readablestream:2278:5)
      at transformStreamDefaultControllerEnqueue (node:internal/webstreams/transformstream:486:5)
      at TransformStreamDefaultController.enqueue (node:internal/webstreams/transformstream:306:5)
      at Object.flush (webpack-internal:///(rsc)/./node_modules/.pnpm/@ai-sdk+openai@0.0.33_zod@3.23.8/node_modules/@ai-sdk/openai/dist/index.mjs:418:24)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at transformStreamDefaultSinkCloseAlgorithm (node:internal/webstreams/transformstream:569:5)
      at Object.close (node:internal/webstreams/transformstream:366:14)
      at ensureIsPromise (node:internal/webstreams/util:185:19)
      at writableStreamDefaultControllerProcessClose (node:internal/webstreams/writablestream:1142:28)
      at writableStreamDefaultControllerAdvanceQueueIfNeeded (node:internal/webstreams/writablestream:1222:5)
      at writableStreamDefaultControllerClose (node:internal/webstreams/writablestream:1189:3)
      at writableStreamClose (node:internal/webstreams/writablestream:699:3)
      at writableStreamDefaultWriterClose (node:internal/webstreams/writablestream:1071:10)
      at writableStreamDefaultWriterCloseWithErrorPropagation (node:internal/webstreams/writablestream:1063:10)
      at node:internal/webstreams/readablestream:1517:15
      at complete (node:internal/webstreams/readablestream:1396:9)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
    issues: [ [Object] ],
    addIssue: [Function (anonymous)],
    addIssues: [Function (anonymous)],
    errors: [ [Object] ]
  },
  value: undefined
}
louis030195 commented 3 months ago

not clearly written in docs that perplexity provider does not support object/json mode

louis030195 commented 3 months ago

assuming it's a bottleneck on their api side?

louis030195 commented 3 months ago

got an answer from them image

dmladenov-dev commented 3 months ago

cause: ZodError: [ { "code": "invalid_type", "expected": "string", "received": "null", "path": [ "knowledge", "certificates", 0, "url" ], "message": "Expected string, received null" },

Got the same error, how can I correct it. Im fine with getting null
lgrammel commented 3 months ago

@dmladenov-dev it looks like your zod schema needs .string().nullish()

dmladenov-dev commented 3 months ago

@dmladenov-dev it looks like your zod schema needs .string().nullish()

Of course 😅. My mistake, thanks for the quick reply!

dmladenov-dev commented 3 months ago

@lgrammel Okay it worked with the nullish but in fact I do need it to be not null. An empty string is what I would like to get as a result if it does not have some information. Is there any way I can enforce that in some way other than the prompt, because there it is not consistent

lgrammel commented 1 month ago

Just did another round of tests. Perplexity does not return structured outputs with JSON or tool modes. Their API just returns text. https://docs.perplexity.ai/reference/post_chat_completions

lgrammel commented 1 month ago

Duplicate of #2378