drizzle-team / drizzle-benchmarks

41 stars 12 forks source link

Are the benchmarks really unbiased? #8

Open lennybakkalian opened 1 week ago

lennybakkalian commented 1 week ago

When comparing prisma-server-node.ts and drizzle-server-node.ts, it seems that Drizzle is heavily preconfigured to perform better in benchmarks (e.g., connection pooling, logging disabled, prepared statements enabled).

I don't have deep knowledge of either framework, but this setup feels quite biased, as one ORM client is fine-tuned while the other appears to be used with its default configuration. Was there a genuine effort to ensure a fair comparison here?

AndriiSherman commented 5 days ago

Having APIs in Drizzle that are not available in Prisma does not mean we should avoid using them. The purpose of benchmarks is to test the fully optimized usage of both libraries and showcase how much more you can achieve with them. Failing to demonstrate this means we would only be testing a fraction of the available features and reducing the scope to raw SQL or database performance tests, where we should be evaluating the features and capabilities of ORMs themselves.