search

drizzle-team / drizzle-orm

Headless TypeScript ORM with a head. Runs on Node, Bun and Deno. Lives on the Edge and yes, it's a JavaScript ORM too 😅
https://orm.drizzle.team
Apache License 2.0
22.08k stars 511 forks source link

[BUG]: Callstack exceeded for large inserts #1702

Open GentikSolm opened 7 months ago

GentikSolm commented 7 months ago

What version of drizzle-orm are you using?

0.29.2

What version of drizzle-kit are you using?

0.20.6

Describe the Bug

SQL for inserts with multiple values are generated via recursion, the call stack can be quickly exhausted with large enough inserts. This particularly becomes an issue when dealing with batched data, or seeding large sums of data into a database. Instead of a separate reproduction repo, I went ahead and added a planetscale example to the drizzle-orm repo here. To see the bug in action, go to examples/planetscale and run pnpm seed after adding pscale credentials for a drizzle connector & installing the packages. (you may need to add the folder to the pnpm workspace to get it to behave properly).

On my machine, this is happening at 100k records, although I have reports that this is also happening with around 10k records on aws lambdas.

Expected behavior

Callstack should not be exceeded

Environment & setup

No response

GentikSolm commented 7 months ago

This is the current execution flow, for documentation. This is most likely obvious to the maintainers but not to fresh eyes on the codebase. recursion is happening from drizzle-orm/src/sql/sql.ts class function buildQueryFromSourceParams image