Support for getMaxAge() or getRemainingMaxAge() ?

[mesonoxian]

Would it be useful to have method getMaxAge() or getRemainingMaxAge() ?. We have all the data to calculate the remaining maxAge.

Any Thoughts?

[andyfischer]

I would use this function if it was builtin, but I worked around it by storing the item's set-time as part of its value, and checking the value with peek.

[mako-taco]

would be useful to implement something like 'stale-while-revalidate'

[ckeeney]

+1 for 'stale-while-revalidate' use case.

It would be trivial for consumers of this library to implement stale-while-revalidate if there was some way to check whether the returned value is actually stale.

[isaacs]

Someone else asked in another issue for a cache.fetch(key, fetchFn) which would return cache.get(key) if it's available and not stale, and otherwise would call fetchFn and then call cache.set(key, fetchFnReturnValue) and return that.

But it sounds like to do stale-while-revalidate, you'd have a fetchFn that returns a Promise, and return the stale value while it's running.

In that case, we'll probably also want to track all the promises in flight, so we're not fetching the same value multiple times, right?

What happens if it falls out of cache while the fetch is happening? Do we still re-insert it?

[isaacs]

Could do something like this:

  async fetch (k, fetchFn, {
    allowStale = this.allowStale,
    updateAgeOnGet = this.updateAgeOnGet,
  } = {}) {
    const index = this.keyMap.get(k)
    if (index === undefined) {
      const p = fetchFn(k).then(v => {
        this.set(k, v)
        return v
      })
      this.set(k, p)
      return p
    } else if (this.isStale(index)) {
      const p = Promise.resolve(fetchFn()).then(v => {
        this.set(k, v)
        return v
      })
      return this.allowStale ? this.valList[index] : p
    } else {
      this.moveToTail(index)
      if (updateAgeOnGet) {
        this.updateItemAge(index)
      }
      return this.valList[index]
    }
  }

So it always returns a promise, which either resolves immediately (if it was found in the cache and not stale, or found in the cache and allowStale is true), or resolves eventually when the fetchFn resolves (if stale and allowStale is false, or if not found).

Calls the fetchFn if it's stale or not found (even if allowStale is true), and stores the promise in the cache (so subsequent calls to cache.fetch(key) will not kick off the fetchFn again). When the promise resolves, it replaces the cached promise with the result that it resolved to.

The weird thing is that it means cache.get(key) will be returning a promise for a while, and then magically start returning a real value. That feels a little magic-at-a-distance. Maybe it should just cache the promise (or whatever fetchFn returns)? That's a lot simpler:

  fetch (k, fetchFn, {
    allowStale = this.allowStale,
    updateAgeOnGet = this.updateAgeOnGet,
  } = {}) {
    const index = this.keyMap.get(k)
    if (index === undefined) {
      const p = fetchFn(k)
      this.set(k, p)
      return p
    } else if (this.isStale(index)) {
      const p = fetchFn()
      const v = this.allowStale ? this.valList[index] : p
      this.set(k, p)
      return v
    } else {
      this.moveToTail(index)
      if (updateAgeOnGet) {
        this.updateItemAge(index)
      }
      return this.valList[index]
    }
  }

But then in both of these, if you call await cache.fetch(key) twice, and the promise hasn't resolved, you don't get the stale value twice, which is what I'd expect. You get the stale value once, and a long wait the second time.

This feels useful, but it's a little tricky to figure out how to make it work in a way that's sensible with the other cache semantics.

[isaacs]

Another pass: https://github.com/isaacs/node-lru-cache/commit/c07a1e48b4494acfd62e42de30ca074b44d78aea

So this will return the fetchFn's promise if you don't allowStale. But if you do, it'll stash the old stale value as __staleWhileFetching on the promise, and once it returns, replace it in the cache and update the start time so it is no longer stale.

Then in get(), if you allowStale, it'll keep returning that stale value while updating.

[isaacs]

Example of using it: const LRU = require('./')

const cache = new LRU({
  ttl: 100,
  max: 3,
  allowStale: true,
})
const fetchFn = (k) => new Promise(res =>
  setTimeout(() => res([k, Date.now()]), 1000))

const keys = ['apple', 'banana', 'cafe']
let k = 0
let n = 1000
const f = async () => {
  k = (k + 1) % keys.length
  const p = cache.fetch(keys[k], fetchFn)
  console.error(await p)
  if (n --> 0) setTimeout(f, 100)
}

f()

The output hangs for a few seconds for the initial fetch, and then flies by like this:

[ 'banana', 1644429098864 ]
[ 'cafe', 1644429099980 ]
[ 'apple', 1644429101084 ]
[ 'banana', 1644429098864 ]
[ 'cafe', 1644429099980 ]
[ 'apple', 1644429101084 ]
[ 'banana', 1644429098864 ]
[ 'cafe', 1644429099980 ]
[ 'apple', 1644429101084 ]
[ 'banana', 1644429098864 ]
[ 'cafe', 1644429099980 ]
[ 'apple', 1644429101084 ]
[ 'banana', 1644429098864 ]
[ 'cafe', 1644429099980 ]
[ 'apple', 1644429101084 ]
[ 'banana', 1644429102188 ]
[ 'cafe', 1644429102289 ]
[ 'apple', 1644429102391 ]
[ 'banana', 1644429102188 ]
[ 'cafe', 1644429102289 ]
[ 'apple', 1644429102391 ]
[ 'banana', 1644429102188 ]
[ 'cafe', 1644429102289 ]
[ 'apple', 1644429102391 ]
[ 'banana', 1644429102188 ]
[ 'cafe', 1644429102289 ]
[ 'apple', 1644429102391 ]
[ 'banana', 1644429103413 ]
[ 'cafe', 1644429103515 ]
[ 'apple', 1644429103619 ]
[ 'banana', 1644429103413 ]
[ 'cafe', 1644429103515 ]
[ 'apple', 1644429103619 ]
[ 'banana', 1644429103413 ]
[ 'cafe', 1644429103515 ]
[ 'apple', 1644429103619 ]
[ 'banana', 1644429103413 ]

So the same stale value keeps getting returned, then updated when the fetch returns.

[MartinKolarik]

But if you do, it'll stash the old stale value as __staleWhileFetching on the promise, and once it returns, replace it in the cache and update the start time so it is no longer stale.

That's neat, I recently implemented something similar and went with the "You get the stale value once, and a long wait the second time." semantics - might borrow this solution.

Small note here, it might be useful to pass the stale value to fetchFn() as a second argument - e.g. if it does an HTTP request internally, the stale value can be used for conditional revalidation.