Can't handle large schemas?

[rossPatton]

I have been using koa-zod-router in my API for about 6 months. Until now, I've had no issues with it.

We are reworking a form that submits to a koa-zod-router endpoint. Previously, this form had a schema with around 30 keys on it.

Now, it has over 50. A lot of these new keys are optional/conditional followup questions that have been added in the new form design.

This schema triggers an infinitely deep error from Typescript. Too much type inference. But only when passed to koa-zod-router for validation.

In an attempt to deal with this, I dramatically simplified the schema itself. Enums became strings, etc. I was also initially building up the schema by merging in sub-sections of the form (sub schemas) and then using the built up schema for final validation. I now just pass in a hard-coded schema.

This solves the infinitely deep problem. Type inference works, but is now incredibly slow. Borderline unusably slow.

In addition - instead of returning an VALIDATION_ERROR, it now seems to allow the request to just go through, but tacks on an issues array to the request body.

What about passing in a schema to koa-zod-router would causes this behavior?

[JakeFenley]

Not really sure what to suggest here.

Under the hood the library is using zod's validation functions to parse objects. so the larger the request body the harder on performance its going to be, no real way around this other than excluding validation.

As for type inference, maybe you can type out your zod schemas and create types based off them and pass them parameters in your route definition. and performance will be better? idk. The thing is in order for this library to work theres multiple levels of type inference it has to go through in order to by compliant with koa-router and zod types. the library basically infers things from mapping the generic types off the named keys of the validation object. so maybe by defining them explicitly eases the load on the typescript server?

I.e. const bodyValidation = z.object({ id: z.coerce.number() }) type Body = typeof bodyValidation zodRouter.get<ZodTypeAny, ZodTypeAny, ZodTypeAny, ZodTypeAny, Body, ZodTypeAny, ZodTypeAny>(....params)

the type parameters are <State, Headers, Params, Query, Body, Files, Response> in that order, in every route/middleware function.

In addition - instead of returning an VALIDATION_ERROR, it now seems to allow the request to just go through, but tacks on an issues array to the request body.

not really sure why the json blob size would affect this, would probably need some code samples to help you with this.

hope this helps

[JakeFenley]

@rossPatton Did you manage to resolve this?

[rossPatton]

Hey sorry,

Yes, but not in a great way. I basically had to keep reducing the complexity (and accuracy) of the schema until it worked. Simple types like z.string() over an enum for instance, seemed to cause less inference. In addition, I reversed how I was building the schemas. Instead of merging in sub-schemas, I hard-coded the main schema and then picked from there to create the sub-schemas. This seems to reduce inference a bit as well.

Zod docs mention hard coding the schema type via generics, which I attempted. But that didn't seem to help.

I can try your suggested approach here, but it might be a little while before I get around to it unfortunately

[JakeFenley]

Yeah tbh i had a hard time getting the thing i actually suggested to work because i was curious about it and tried it, so maybe dont bother with that heh. Maybe it is soemthing to do with the enum types then because I had tried to replicate your problem with large schemas but only with regular primitive types nested in an object, which did give a lot of latency in type inference once the schema became absurdly large but it didnt necessarily break. maybe it is something to raise on the zod github?

its hard from my point of view to correct something like this, because really the library is dependant on a number of things such as how koa and koa-router generics are typed, and how zod's type generics work. in terms of types the library is mostly just a middleware for bridging those gaps. it is frustrating that it isn't scaling for you, and i had always feared this in a large scale application.

I am sorry that this probably isnt very helpful, i'd like to be able to just come up with a fix and create a patch version for this, but i honestly don't know what to do here.