Accessing GraphQL extensions using the JavaScript react-apollo client

[dwwoelfel]

OneGraph offers an easy and consistent GraphQL interface to dozens of both fun and critical APIs. As we’ve built our integrations and infrastructure, we’ve become deeply familiar with so many of the opportunities to push incidental complexity down into the platform. That way developers get to focus on the far more interesting challenges. But when we’re building OneGraph itself, we often want to know how well we’re doing - for example, given how powerful GraphQL is, and how much data a developer can pull out with a single query, we wanted to know:

What level of optimizations are we making for a given query over the simple, direct REST alternative a developer would write?

At OneGraph, we implement quite a few clever batching and caching strategies, tailored for the services that our users are querying or mutating against. These strategies involve years of experience working with APIs of all shapes, combined with a deep understanding of the underlying APIs we’re interacting with.

Here's an example query that shows how effective it can be!

How we’re able to optimize query execution against REST apis

To take one example in particular, the Trello API has a very powerful batching feature in their API, where with a single call you can retrieve several resources.

The batch endpoint allows you to make multiple GET requests to the Trello API in a single request. By batching GET requests together, you can reduce the volume of calls you are making to the API and more easily stay within your API rate limit. The batch endpoint can not be called recursively; requests containing the batch url will be ignored. [from Trello’s (slick) API docs]

API-specific optimizations: Batching

To make use of Trello’s specific batch api, you hit the batch REST API endpoint with a list of other REST urls you want to retrieve. Because of GraphQL’s declarative nature, OneGraph’s is able to walk a developer’s query at execution time and determine the optimum strategy - if we see that a board is queried for, and then all of its cards, and then the comments for those cards, we can build that up into a single API call, rather than the several hundred calls it would be with a naive implementation. We also make sure to only make a request for a given resource one time, even if the query recursively refers to it.

API-agnostic optimizations: resource de-duping

Recursive references to the same resource are quite common actually! Take for example a query into Trello that traverses board->lists→cards→board. The first board reference might pull out information that’s best displayed (and therefore accessed) at the top of the UI, but we might want to have some different fields from the board when rendering the cards. This gives the GraphQL consumer additional power to specify where in the response they want those fields (and therefore reduce the effort to reshape the data in code after getting the response). But a naive implementation may end up hitting the same endpoint dozens (or even hundreds!) of times.

API-agnostic optimizations: per-request caching

Batch queries on Trello are still limited to 10 routes, and it’s very likely that we’ll need more than that to complete a full query for our users. So after completing the first call to the batch api, we store the results in a local in-memory cache to be used only for that query. That cache will keep track of the result from each route. As we build up any future calls to Trello resources, we check our cache to see if we already have it in memory, thus further reducing upstream calls.

Additional benefit: Per-request consistency

We automatically parallelize the execution of any query on OneGraph to the maximum degree possible by the dependencies in the graph and the rate the upstream API can handle. But a large query across many services may take a second or two to complete - and those several seconds are a chance for some data to change in upstream resources.

One overlooked benefit to per-request caching is that any resource that’s referenced multiple times in one query will appear consistent, which really helps reduce some bugs when it comes to rendering.

Putting it all together: Great for the user, Great for the developer, Great for The API

This is wonderful for the end user because it reduces total time fetching resources, meaning speedier apps and lower battery usage.

For developers it’s wonderful because it reduces their chance of getting rate-limited as well as the chance of inconsistencies in resources inside a single query

And it’s wonderful for the API-provider because it gives them a much more cohesive view of their users’ needs (what data is requested and used together) while also reducing the load on their servers as much as possible.

Prior art

The challenges and techniques we outlined above are spiritually similar to Facebook's DataLoader

Continuously measuring the wins

For our end-users, it’s possible to get a sense of how effective our strategies are for any given query. When you add show_metrics=true to the requests url, we’ll include the results alongside data and errors under the extensions key.

https://serve.onegraph.com/dynamic?app_id=${appId}

becomes

https://serve.onegraph.com/dynamic?app_id=${appId}&show_metrics=true

That’s it! To compare, take this Trello query:

query whatIsSpolskyWorkingOn {
  trello {
    member(username: "spolsky") {
      fullName
      boards {
        nodes {
          name
          cards {
            nodes {
              name
              list {
                name
                board {
                  name
                }
                cards {
                  totalCount
                }
              }
            }
          }
        }
      }
    }
  }
}

The results with metrics will look something like

{
  "data": "...",
  "extensions": {
      "metrics": {
          "api": {
              "avoidedRequestCount": 149,
              "requestCount": 5,
              "totalRequestMs": 1384,
              "byHost": {
                  "api.trello.com": {
                      "requestCount": 5,
                      "totalRequestMs": 1384
                  }
              }
          }
      }
  }
}

We even use this to power our GraphiQL instance’s reporting:

5 API requests instead of 154! That level of optimization feels great to see, and even better when it comes with no additional effort 😊

Try it out with this query for yourself!

Reporting our optimization performance to the end developer

Following our GraphiQL Explorer open source release, we have an upcoming app built on top of OneGraph and several APIs that have become very useful for us internally. We wanted to communicate to both us and the end user how effective our strategies were so that we can find even more opportunities to improve!

Our app is built using a rather nice set of tools, including Zeit’s next.js and react-apollo. The challenge however, is that react-apollo doesn’t provide a great way to communicate the extensions of individual GraphQL responses to React components.

There isn’t a great way for react-apollo to communicate the results of individual API calls to components

This makes sense because of Apollo’s caching implementation, where components never access response data directly, but instead access a normalized cache of data that has been populated by the responses. The implementation is really rather interesting to read through and understand, and points to some of the potential strengths of GraphQL’s design to push down complexities into the system in the future. But it means we need to find a nice way to access extensions data on our own.

Custom ApolloLink

Our approach used Apollo’s equivalent of middleware to intercept the response data as it’s returned from the server, but before it’s inserted into the cache and the components re-rendered. We use a bit of mutable state to track each request and its result, and the use that state to render the metrics inside the app. Now on each request we can include notes about how effective our strategies are!

The many, varied uses of extensions

In our case we want to understand how effective our optimizations are inside of the OneGraph service, but there are many other use-cases for extension data! One great example is passing tracing data the resolution time of individual fields to show a waterfall view of a query’s performance - hugely useful for understanding what data slows down your rendering, and telling you how to split up an app’s data dependencies to speed things up.

Tradeoffs of a Custom ApolloLink Solution

This solved a number of challenges that come with Apollo’s caching design, but it present several tradeoffs. First, it involves diving a bit deeper into the implementation details of Apollo, and in general it’s best to be able to shield the developers who are consuming your GraphQL API from the boilerplate to connect to services. And second, the implementation is non-deterministic, and means that there can be some slight race conditions between the data that’s rendered and the data that’s in the metrics cache. In our case, the extensions are extremely useful, but ancillary information, so we felt the tradeoff was acceptable, but you’ll have to make the call for your own app, of course.

Source code and example

Check out the source code for our example to see how you can access extension data in your react-apollo app and make your developer’s lives better today!

{"source":"blog","publishedDate":1549008000000,"url":"https://www.onegraph.com/blog/2019/02/01/Accessing_GraphQL_extensions_using_the_JavaScript_react-apollo_client.html"}