Allow returning stale values while caches are revalidated

[reaktivo]

TLDR: Support a staleWhileRevalidate hint on the cacheControl directive

type Variant {
  name: String
  sku: String
  inStock: Boolean @cacheControl(maxAge: 5, staleWhileRevalidate: 100)
}

I've been experimenting with cacheControl hints on an Apollo Server. The current per field cache control that Apollo exposes is quite powerful. It has a real impact on the response times of our queries.

Saying that, I still have a few REST data sources that are quite slow and every time my cached field value expires, I can notice how the REST endpoint is being hit and the cache recreated. This is of course, expected behaviour.

I was wondering if it would be possible to have a @cacheControl directive hint that would instruct the cache to default to returning cached data even when it's stale and in that specific case, to update the key's value on the background.

This would mean that only the first uncached request would have to wait for the rest endpoint to respond.

From the look of it, it seems that this specific use case would be impossible to implement without both a custom KeyValueCache and a custom apollo-server-plugin-response-cache implementation.

I can see two modifications being required, the first one on the Interface exported by KeyValueCache, having a method in the following form would greatly simplify implementing revalidation on the background. This would give the cache the responsibility of creating the cached value when and if required.

const value = await cache.getOrSetOnStale('key', async () => {
  const updatedValue = await generateValue();
  await cache.set('key', updatedValue);
  return updatedValue;
});

The second change would require having apollo-server-plugin-response-cache pass in the hint that the value should be revalidated in the background if it's stale.

Reference http stale while revalidate implementation

For some reference, see this article: https://www.fastly.com/blog/stale-while-revalidate-stale-if-error-available-today

[timsuchanek]

In case you don't want to wait for this (it's already 1.5 years since you created the issue), here a shameless plug: I'm currently building https://graphcdn.io, a dedicated GraphQL CDN, which has this built-in and configurable. It's compatible with Apollo Server, no code changes required.

[matchu]

Lol we came to this with similar timing!! I hacked together a fork of the Apollo cache control plugin, for apollo-server@2.19.2.

I haven't tested it very extensively, mostly just that it seems to work on first glance! I'm shipping it to production now, we'll see how it goes! 🤞 But maybe other folks who are looking for lo-fi solutions might find it useful!

https://gist.github.com/matchu/6955541b744dc0c313fd3e865cdbb39d

[stewartmcgown]

From what I understand, stale while revalidate at the plugin level is completely impossible.

I've recently tried to combine the above gist and https://github.com/thematters/apollo-response-cache/ to create the perfect SWR response cache, but the lack of the ability in apollo-server itself to dispatch a refreshing task to update the stale data while still serving a response puts a brick wall in front of that.

Unless I'm not understanding the lifecycle methods correctly, responseForOperation expects a binary outcome. Null if there is no cached data, { data: ... } if there is.

Does someone with deep understanding of Apollo internals know if there is a way to essentially 'duplicate' a requestContext and redespatch it to Apollo? This redespatch method could be called within the responseForOperation hook, synchronised via the cache so only one instance of the refresh is called across many apollo-server instances as they continue to serve stale responses.

My attempt is here https://github.com/stewartmcgown/apollo-response-cache

@abernix ? Apologies for the ping but it would be good to know if this is a possibility within Apollo internally before we continue with our plugin

[matchu]

(Ah yeah, you probably already understand this but I should say it out loud for clarity: my gist only implements the HTTP Cache-Control header, and it relies on a separate cache layer like Varnish, Cloudflare, Fastly, or Vercel's built-in cache, to actually perform the response caching according to what the Cache-Control header says!)

[stewartmcgown]

Yep! Which is why I combined it with response-cache-plugin so it would work with existing clients using the POST /graphql verbage. When switching to automatic persisted questions over GET for clients automatic CDN caching works great, but I wanted to use it with a response cache plugin.

[glasser]

@stewartmcgown You're correct that the request pipeline doesn't offer way to do "stale while revalidate" (ie, to branch off the rest of execution into something used only to fill the cache). I think that would be a pretty interesting improvement though; I'd be interested in reviewing a PR (or probably a design doc explaining what the changes would look like first).

You could I suppose have your plugin actually call into processGraphQLRequest itself in the background, though that seems somewhat clunky/overkill.

[stewartmcgown]

Appreciate the advice! Could you elaborate on the processGraphql call please?

[glasser]

It's technically the entry point to the request pipeline. Though it's not currently exported; you could go all the way up to runHttpQuery but that seems like you'd be redoing even more work.

[stewartmcgown]

@glasser Thanks for this. I've begun implementing the SWR process based on your advice.

From what I can see, processGraphQLResponse requires a config object that contains stuff like the 'plugins' key. Where can I access the full apollo server config from within my plugin?

[glasser]

I don't believe it's provided directly; you could pass it to your plugin yourself?