Ability to generate and cache "batches" of items at once

[EvanK]

My rationale

I'm doing a ton of cache operations where I have a list of values ("id"s in the parlance of the policy api) for which I'm generating values with a database query.

Design-wise, I have two clear options:

• "generate" and cache a value for each id separately
  • pro: structurally simple
  • con: puts heavy load on both my cache and my database
• "generate" and cache a whole batch of ids at a time
  • pro: for often repeated batches ("1,2,3"), performs very well
  • con: for differing ("4,5,6") or differently ordered ("1,3,2") batches, this performs poorly with many misses

Proposed addition to API

It would be useful to support generating and getting/setting in cache, batches of items at a time. The underlying strategies could optimize for as little network overhead as possible -- eg, catbox-redis could use MGET/MSET and/or pipelining.

I know that operating with a batch of ids and a superset of cache values adds an order of magnitude in complexity, but you could foist that complexity onto the user of the lib by accepting a collection of functions in place of generateFunc:

• batchFunc given a list of ids that missed cache (or have become stale?), generate a superset of values (for all the given ids)
• filterFunc given that superset of all newly generated values and an id, filter out values not relevant for the given id (to then set in cache under the given id)
• mergeFunc given multiple values (some from cache, some freshly generated), combine them somehow for catbox to return as a batch

So the proposed batching API would be something like this:

options = {
  // same options we have now
  expiresIn: "1 hour",
  staleIn: "5 minutes",
  // and then something different...
  generateFunc: {
    batchFunc: async (ids) => { return knex('some_table').select('id', 'priority', 'fk_id').whereIn('fk_id', ids) },
    filterFunc: async (collection, id) => { return collection.filter((item) => id === item.fk_id) },
    mergeFunc: async (collections) => { return collections.reduce((accum, value) => accum.concat(value), []) },
  }
}

// new policy with above options
policy = new Catbox.Policy(options, client, `cacheNamespace.`)

// will miss cache, then generate (and set) four separate items for each arg
firstBatch = await policy.batch(1,2,3,4)

// will hit cache for both items, that were set previously
secondBatch = await policy.batch(3,4)

[hueniverse]

I think this is just beyond the scope of this module. This will work better for you if you just write your own cache wrapper around your exact needs.

I'll let @kanongil decide if he wants to reconsider.

[lock[bot]]

This thread has been automatically locked due to inactivity. Please open a new issue for related bugs or questions following the new issue template instructions.