StefanTerdell / zod-to-json-schema

Converts Zod schemas to Json schemas
ISC License
886 stars 71 forks source link

Maximum call stack size exceeded in zodToJsonSchema #94

Closed Rajavasanthan closed 10 months ago

Rajavasanthan commented 10 months ago

I am writing a backend API in Nextjs14

const topicSchema = z.object({
      topics: z
        .array(
          z.object({
            topic: z.string().describe("The topic of the position")
          })
        )
        .describe("An array of topics"),
    });
    const res = zodToJsonSchema(topicSchema)

this code return Maximum call stack size exceeded

RangeError: Maximum call stack size exceeded
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    ...

I am using zod-to-json-schema with NextJS 14, langchain ^0.0.177

StefanTerdell commented 10 months ago

Can't reproduce. Please check if this looks right https://github.com/StefanTerdell/zod-to-json-schema/blob/master/test/issues.test.ts#L6

Rajavasanthan commented 10 months ago

I have re-produced the same issue. I have made a project in Stackblitz please take a look at it.

https://stackblitz.com/edit/stackblitz-starters-foddlw?description=The%20React%20framework%20for%20production&file=app%2Fapi%2Ftest%2Froute.ts&title=Next.js%20Starter

Steps npm install && npx next dev Visit : /api/test

You can see the logs

StefanTerdell commented 10 months ago
it-works
jhiester-shell commented 10 months ago

I am seeing this issue as well. What version of node were used in the repo and your response @StefanTerdell @Rajavasanthan ?

StefanTerdell commented 10 months ago

@jhiester-shell It's 18.18.0 in the playground @Rajavasanthan provided

jhiester-shell commented 10 months ago

can you test on 20.10.0?

StefanTerdell commented 10 months ago

I just tested the provided example on 16.20.0, 18.18.0 and 20.10.0 with node commonjs, node esm and ts-node (10.9.1, defaults only) with no issue. closing until someone can reproduce this.

rheaton64 commented 10 months ago

I have a very similar issue with node 18.18.0 and Next 14. Simple schema, same recursion error as above.

const zodSchema = z.object({
  questions: z.array(
    z.object({
      question: z.string().describe("One question on the test"),
    })
  ).length(5).describe("List of 5 questions on the test"),
});

Using with ai^2.2.20

const functionCallingModel = llm.bind({
    functions: [
        {
        name: "create_test",
        description: "Create a 5 question test. The questions should slowly increase in difficulty.",
        parameters: zodToJsonSchema(zodSchema, "Test"),
        },
    ],
    function_call: { name: "create_test" },
    });

error:

 ⨯ RangeError: Maximum call stack size exceeded
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
    at Module.zodToJsonSchema (webpack-internal:///(rsc)/./node_modules/zod-to-json-schema/dist/esm/index.js:43:107)
...
dominics commented 10 months ago

@StefanTerdell Reproduction is to use the link @Rajavasanthan posted, but change line 7 (the import) to use a named import:

import { zodToJsonSchema } from 'zod-to-json-schema';

Here is an updated StackBlitz. As before, visit /api/test, and now it breaks as described:

image

I suspect ESM weirdness in Webpack in my and @rheaton64's cases (I'm bundling for a Firebase function using Webpack 4), and whatever next is doing in @Rajavasanthan's case (swc, but then Stackblitz doesn't support it, so falls back to Webpack?).

Answer might just be: don't import it like that! But if so, that'd be a BC-break, no? (Unless the project doesn't consider bundled usage as part of API stability?). There's also third-party code I can't control that imports it like this (e.g. fastify-type-provider-zod) so those aren't easy to fix.

The issue test won't pick it up because it's not going through package.json nor bundling.

I think this is why packages usually add an ESM version at the next major version bump (because it's unpredictable which of the exports listed in package.json a bundler will choose, and it's hard to test a package under bundlers.)

StefanTerdell commented 10 months ago

Jesus christ

kachkaev commented 10 months ago

Same issue here when using Next.js 14 with monaco editor. The crash only happens in prod. I am using zod-to-json-schema for autocompletion in monaco. The code looks something like this:

    monaco.languages.json.jsonDefaults.setDiagnosticsOptions({
      ...monaco.languages.json.jsonDefaults.diagnosticsOptions,
      validate: true,
      schemas: [
        ...(monaco.languages.json.jsonDefaults.diagnosticsOptions.schemas ?? []),
        {
          uri: jsonSchemaUri,
          fileMatch: [modelPath],
          schema: zodToJsonSchema(schema),
        },
      ],

It used to work, but started crashing after I updated from 3.21.4 to 3.22.0.

kachkaev commented 10 months ago

This diff might fix it: https://github.com/StefanTerdell/zod-to-json-schema/commit/82d339ecfe0775b697b8e1643451cd3eaf9115c8#diff-a2a171449d862fe29692ce031981047d7ab755ae7f84c707aef80701b3ea0c80L2

Happy to try a canary release and report if I get the same crash in next start as for 3.22.0.

StefanTerdell commented 10 months ago

@kachkaev Sure, I just published 3.22.1@beta

kachkaev commented 10 months ago

It worked! After updating zod-to-json-schema to 3.22.1-beta.0, I can no longer observe a crash in next start 🎉

StefanTerdell commented 10 months ago

Nice. I just released it on @latest tag as well. Keeping this open for a bit in case we get stragglers, but fingers crossed it should be fixed

So I can understand why the recursive export caused an overflow... What I don't get is what it had to do with named/default exports