Open warflash opened 5 months ago
Thanks for the effort you put into this benchmark! I'm broadly aware of this and am working on solutions in Zod 4. There are some very obvious performance in Zod's current parsing pipeline that unfortunately will require breaking changes to fix.
I'll report back with updated performance numbers against this benchmark once Zod 4 is nearing beta.
Great, that sounds very nice! Appreciate the reply and really looking forward to trying out the v4 beta then to see how it performs against our production ingests 😊
Wonder if it would work to get rid of ctx.async
and just handle all results from _parse()
as potentially async (i.e. check whether the return value is a Promise
. Currently there seems to be a lot of potentially unnecessary Promise
wrapping just based on ctx.async
^I made a POC of not using ctx.async
and instead detecting Promises during parsing. Here's the branch https://github.com/jussisaurio/zod/pull/1/files which currently is based on my other zod perf PR (https://github.com/colinhacks/zod/pull/3487)
Results using your benchmark @warflash :
zod@3.23.4:
Benchmarking with 1000 elements
Validation time sync: 25.703ms
Validation time async: 61.169ms
Benchmarking with 10000 elements
Validation time sync: 110.733ms
Validation time async: 641.032ms
Benchmarking with 50000 elements
Validation time sync: 592.054ms
Validation time async: 4.608s
Benchmarking with 100000 elements
Validation time sync: 1.541s
Validation time async: 13.151s
branch:
Benchmarking with 1000 elements
Validation time sync: 20.666ms
Validation time async: 11.56ms
Benchmarking with 10000 elements
Validation time sync: 113.841ms
Validation time async: 100.013ms
Benchmarking with 50000 elements
Validation time sync: 510.81ms
Validation time async: 512.89ms
Benchmarking with 100000 elements
Validation time sync: 1.056s
Validation time async: 1.188s
The reason sync/async are almost the same in the branch is that it doesn't extra wrap any promises. A more comprehensive benchmark would include stuff like async refines to see what the impact is
Running
parseAsync
instead ofparse
seems to have massive differences in resource consumption for larger objects.I'd expect some overhead of async operations for sure, however in our production env. and in the benchmark as well the difference is 100x in time consumption and 10x in memory consumption. Larger objects even OOM the entire node app with
parseAsync
whereasparse
hovers at around 300mb.The reproduction of a minimal bench setup can be found here: https://github.com/warflash/zod-async-perf
Tested on 2 machines with node 18 and 20.
Logs: