Closed naruaway closed 1 year ago
The benchmarks should also take into account that we have deprecated .error
in safeParse
, which will make safeParse
faster than parse
from one of the next versions. Also, it has to be considered whether a validation library collects all issues or aborts at the first one. If I see correctly, the Typia implementation uses .createIs
instead of .createValidate
. Valibot should also use the is
function or the abortEarly
property for a direct comparison.
@fabian-hiller
The benchmarks should also take into account that we have deprecated .error in safeParse, which will make safeParse faster than parse from one of the next versions.
Sure, but all the test cases in typescript-runtime-type-benchmarks is for "valid" data, so throw
will not happen anyway even for .parse
so I think the perf of parse
and safeParse
should not be different. This is also consistent with the test cases for Zod, which can throw when the data is invalid.
Also, it has to be considered whether a validation library collects all issues or aborts at the first one. If I see correctly,
I agree with this but since typescript-runtime-type-benchmarks includes a lot of libs, it's almost impossible to do "complete fair" comparison anyway. Typia cannot have "validation" logic since the compiled output consists of simple checks. I think what we can/should do the best here is trying to align with the case for Zod. And Zod collects all errors. However, all of this should not matter for the benchmark result in typescript-runtime-type-benchmarks since it does not run benchmark for "invalid" data. I guess this might be interesting addition for typescript-runtime-type-benchmarks.
And this is one of the reason why I am experimenting with https://github.com/naruaway/valibot-benchmarks, which includes more diverse schema and invalid data. People are using this typescript-runtime-type-benchmarks for evaluating PRs like https://github.com/fabian-hiller/valibot/pull/104 but we cannot reveal any perf issues for invalid data case. For this specific PR, I'll try to leave a comment later there
cc: @fabian-hiller
.parse()
on the Valibot schema was never intended to be used directly.I did not notice this while I did the change in https://github.com/moltar/typescript-runtime-type-benchmarks/pull/1112 since I was just relying on auto completion based on TypeScript types and
.parse
appeared.Note that the future version of Valibot renamed it to
_parse
to avoid this confusion.Anyway let's use the intended documented API. Otherwise, updating Valibot version in the future will break this test case. Note that this should not affect performance since internally it's just calling the
.parse
on the schema now.