librariesio / libraries.io

:books: The Open Source Discovery Service
https://libraries.io
GNU Affero General Public License v3.0
1.11k stars 203 forks source link

Query for all Dependency IDs Instead of Using Batching SQL Queries #3157

Closed mikeyoung85 closed 1 year ago

mikeyoung85 commented 1 year ago

The query without using the batching SQL query parameters, namely order by id asc where id > ? limit 1000, seems to run pretty quickly and should avoid timing out. This change may result in the container hitting some memory issues, but that seems more manageable than trying to speed up the queries on these tables some more.