volatiletech / sqlboiler

Generate a Go ORM tailored to your database schema.
BSD 3-Clause "New" or "Revised" License
6.56k stars 533 forks source link

Improve loads by using maps for deduplication #1337

Closed nicowolf91 closed 6 months ago

nicowolf91 commented 6 months ago

Let maps handle the deduplication logic to increase performance / improve response times.

Commit Deduplicate IDs in eager loads. introduced nested loops to filter duplicate foreign key IDs that are passed to the WhereIn-Function. We noticed that the nested-loop-implementation of the deduplication logic results in bad performance as the number of eagerly loaded entities increases: #267.

We considered completely dropping the deduplication since databases treat IN clauses as sets. The map-implementation improves performance and keeps the resulting SQL statements "backwards-compatible".

Please let us know if anything is missing!