Initial implementation of cache control

martinbonnin commented 2 years ago

When using the normalized cache, it should be possible to expire data:

either declaratively or programmatically from the client
or using server headers if any

There are currently very little options for this

HttpCache has "CACHE_EXPIRE_TIMEOUT_HEADER" which is client controlled and doesn't read the server value
MemoryCache has "expireAfterMillis" which is client controlled and doesn't read the server value (and works at the Record level)
SqlNormalizedCache has no expiration

tentative API

By default, the maxAge should come from the server. Given this response:

200 OK
Cache-Control: max-age=60
...
{
  "data": {
    "hero" {
      "name": "Luke"
    }
  }
}

A client requesting this data after 60s will fail from cache and fallback to the network:

// Initial network request
val response1 = apolloClient.query(HeroQuery()).fetchPolicy(NetworkOnly).execute()
assertFalse(response1.isFromCache)

// This will come from cache
val response2 = apolloClient.query(HeroQuery()).fetchPolicy(CacheFirst).execute()
assertTrue(response2.isFromCache)

delay(60_000)

// Cache has expired, fallback to network
val response3 = apolloClient.query(HeroQuery()).fetchPolicy(CacheFirst).execute()
assertFalse(response3.isFromCache)

It should be possible to override the maxAge from the client when getting the data:

val response1 = apolloClient.query(HeroQuery())
       .fetchPolicy(NetworkOnly)
       .overrideMaxAge(0) // expire immediately
       .execute()

Or programmatically on the store itself:

// expire immediately
apolloStore.expire(HeroQuery(), System.currentTimeMillis/1000) 

// expire in 10s
apolloStore.expire(HeroQuery(), System.currentTimeMillis/1000 + 10) 

// The two methods above set a fixed point in time when to expire but it can also be interesting to set just the maxAge
apolloStore.setMaxAge(HeroQuery(), 10)

Because some stale data is better than nothing, the cache should be configured with a maxSize and not remove entries immediately:

ApolloClient.Builder()
     .normalizedCache(SqlNormalizedCacheFactory(
        name = "apollo",
        maxSizeMB = 100_000_000,
    ))
    .build()

This way, a client can use stale data if it wants to:

// Always display stale data if it's there
apolloClient.query(HeroQuery()).acceptStaleDataFor(Long.MAX_VALUE).executeCacheAndNetwork().collect {
}

Client-side expiration logic

If the server didn't send any cache information, we could think of "hardcoding" the rules in the client:

extend type Query {
  currentMinute @maxAge(60)
}
extend type Day @maxAge(86400)

Or that could be a runtime thing too:

/**
 * coordinates is a schema coordinates as in https://github.com/graphql/graphql-spec/issues/735
 */
class ExpirationInfo(val coordinates: String, val maxAge: Long) 

fun ApolloClient.Builder.normalizedCache(
      normalizedCache: MemoryCacheFactory,
      expirationInfo: List<ExpirationInfo>
)

I'm not too sold on that idea yet because that adds friction between the client and the server but that'd be a way to communicate per-field cache information out of band.

Related work:

Server side @cacheControl

Part of apollographql/apollo-kotlin#2331

BoD commented 2 years ago

About the directive: would it make sense for it to be @cacheControl(maxAge: 60) to be like the Apollo Server one - or maybe we don't want to on purpose (or can't)?

martinbonnin commented 2 years ago

When discussing the declarative cache directives, we decided to not reuse the federation @key and go for @typePolicy instead as we were not sure the semantics would be exactly identical.

I think it's safer to do that assumption here as well.

That being said, I don't have any strong opinions about naming at this stage. This was more of a placeholder than anything else.

BoD commented 2 years ago

Oh interesting! Makes sense!

martinbonnin commented 2 years ago

Updated the initial description after the https://github.com/apollographql/apollo-kotlin/issues/3709 discussion.

martinbonnin commented 2 years ago

https://github.com/apollographql/apollo-kotlin/pull/4104 and https://github.com/apollographql/apollo-kotlin/pull/4156 introduce a new (experimental) SQLite cache backend (named Blob) that can store a date alongside the fields. This can be used to implement both client driven expiration based on the received date and server driven expiration based on the “Cache-Control" header.

Client driven:

    val client = ApolloClient.Builder()
        .normalizedCache(
            normalizedCacheFactory = SqlNormalizedCacheFactory(name = fileName, withDates = true),
            cacheKeyGenerator = TypePolicyCacheKeyGenerator,
            cacheResolver = ReceiveDateCacheResolver(maxAge)
        )
        .storeReceiveDate(true)
        .serverUrl("https://...")
        .build()

Server driven:

    val apolloClient = ApolloClient.Builder()
        .normalizedCache(
            normalizedCacheFactory = SqlNormalizedCacheFactory(name = fileName, withDates = true),
            cacheKeyGenerator = TypePolicyCacheKeyGenerator,
            cacheResolver = ExpireDateCacheResolver()
        )
        .storeExpirationDate(true)
        .serverUrl("https://...")
        .build()

Cache resolver API

Both these APIs leverage the CacheResolver API that has access to both the __typename and the cache date so it should be reasonably doable to implement custom logic there.

Follow up

I'm going to close this issue as it was quite broad and follow up in more focused ones:

theBradfo commented 2 years ago

@martinbonnin follow-up question on this, If i were to use the NormalizedCache direclty, without using ApolloClient would it be possible to somehow use this new CacheResolver api with the direct usage of NormalizedCache itself?

martinbonnin commented 2 years ago

I would say so. The CacheResolver API is used by readFromCache so nothing is preventing you to use that without an ApolloClient.

All in all, I like to think of the CacheResolver API as a "mini-backend" that is embedded in the client and can serve data from an unstructured key-value store.

One question you will bump into if you're planning to use the NormalizedCache directly is how to handle concurrency. Right the ApolloStore is handling that. If you're using NormalizedCache directly, you'll certainly have to come up with something. It might be as simple as a big lock but it's something to keep in mind.

apollographql / apollo-kotlin