endlessm / azafea

Service to track device activations and usage metrics
Mozilla Public License 2.0
10 stars 2 forks source link

Implement chunked queries #57

Closed bochecha closed 4 years ago

bochecha commented 4 years ago

Most management commands operate on data sets too large to be fetched all at once and then iterated on without being OOM-killed.

Until this change we wrote custom query chunking for each such command.

We're going to add more, and copy-pasting this code gets old real quick.

This change brings a new dbsession.chunked_query(…) method, and moves the existing management commands to use it.