search

hoover / snoop

Other
6 stars 3 forks source link

move `Digest.updated_at` to `Document.digested_at` #36

Closed mgax closed 7 years ago

mgax commented 7 years ago

The feed query is making a subquery that gets all IDs for a given collection, and then sorting all the updated_at values, which is slow if the collection is large. This patch moves the timestamp into the documents table and creates an index for (collection, digested_at), to speed up feed queries.

mgax commented 7 years ago

Here's a quick benchmark with a synthetic dataset of one million empty digest records. We're consuming the first 1000 items in the feed.

The migration itself took 2 minutes.