Apollo GraphQL: What is "ResponseNormalizer" | Performance issue with client side cache

[rushilpaul]

Question

I see that there is something called ResponseNormalizer in the Apollo GraphQL library for Java.

But I can't find any official documentation for it.

I'm trying to find ways to improve the latency of the GraphQL cache in a Java client that's using the latest Apollo GraphQL Java libraries (backed by Kotlin).

Performance issue:

I have a GQL schema that has a main node called offers which is an array of a Type called Offer which contains many fields.

type Query {
    offers(p1: type1, p2: type2, p3: type3): [Offer]
}
type Offer {
...
}

input p1 {
...
}

input p2 {
...
}
...

Currently, when a cache hit happens, I'm experiencing latencies that are proportional to the number of records returned from the offers array.

This is going up to 30ms with the current size of data and will only get worse as data increases.

I believe this is because of the way cache record normalization / denormalization happens while assembling a response to serve a query.

Questions

1. Is it possible to customize this normalization / de-normalization using ResponseNormalizer mentioned above (or any other way to do this) ?
2. Is it possible to build such a Cache Key resolver that normalization gets minimized? Or should I just use HttpCache in this scenario? I still want to be able to cache responses based on the parameters on the offers node.

[martinbonnin]

Is it possible to customize this normalization / de-normalization using ResponseNormalizer mentioned above (or any other way to do this) ?

Not too much. There's a cost to normalization. We do have some benchmarks here. If you notice your data is not matching, please share a sample response and we'll add that to the benchmarks.

Is it possible to build such a Cache Key resolver that normalization gets minimized? Or should I just use HttpCache in this scenario? I still want to be able to cache responses based on the parameters on the offers node.

HTTP cache will save you the normalization cost but of course it's all a tradeoff. If your app is using watchers, you're now losing that functionality. It might or might not be important. You can read more about this here

Out of curiosity, what's your Java app like? We have a bunch of Java users but most of the time, they are on the backend (Spring mostly) and do not benefit much from client side caching.

[rushilpaul]

@martinbonnin I had a look at the data you shared. Seems like the following would be relevant to me because we're using an in-memory cache:

com.apollographql.apollo3.benchmark.CacheTests.cacheOperationMemory | 59105136 nanos (59.1 ms) com.apollographql.apollo3.benchmark.CacheTests.cacheResponseMemory | 52749770 nanos (52.75 ms)

According to my interpretation of the metrics from the datadog link, if the cache is of size 3.32 MB, it takes 350-360 ms to read data from the cache.

In my own tests, the cache size was around 140 KB and it was taking around 16 ms which is close to the 14 ms of latency I extrapolated for 140 KB cache size from your test data.

And I think my questions are answered here. Thanks!

Out of curiosity, what's your Java app like?

We have a backend GraphQL based service that responds with data (cached at the server) in the schema mentioned above (cannot copy the entire schema here). We have built a thin java client that calls our GQL API. There are several clients who have integrated with our service using this thin client. A few of these clients are very latency sensitive and can't afford more than a few milliseconds of latency on their downstream client invocations.

[rushilpaul]

Do you have benchmark metrics for ApolloHttpCache as well?

[martinbonnin]

Thanks for the details! So I'm assuming these clients are CLI/Desktop apps?

We don't have data for HTTP cache unfortunately but I expect it to be a lot faster since it skips the normalization phase.

Another option you'd have is cache the data in memory using a simple in-memory LruCache. This will be the faster you get.

[BoD]

Closing for now - don't hesitate to open a new issue about this if needed.

[rushilpaul]

@martinbonnin

Thanks for the details! So I'm assuming these clients are CLI/Desktop apps?

They are AWS apps. Our GQL server is also running in AWS in a separate account but these are all intra-region calls.

Another option you'd have is cache the data in memory using a simple in-memory LruCache. This will be the faster you get.

This is what we ended up doing eventually! :) Got us down to < 1 ms response regardless of the cache size because we're now just returning object references to our upstream client upon a cache hit.

However, the downside of this is that it is not a "thin client" anymore - There are inherent risks of using a third-party in-memory cache (Google Guava in this case). I see the solution as a bit fragile, since I'm keying the cache on the query doc + query variables and I'm not sure whether it can change format etc. We will have to live with it for some time until ApolloGraphQL provides some features around this in future.