Any examples of using a database as a datasource rather than a REST api?

[JacobT14]

I can't seem to find anywhere any examples(or the library to use) to accomplish the datasource concept through a database library(i.e. sequelize). I switched over from graphqlLambda to Apollo Server 2 and now my responses are taking 6-10 times as long(300ms --> 2.2-2.8 seconds) and I'm thinking maybe it is because I'm not including a datasource?

[linonetwo]

You can try prisma-binding with apollo-server.

[JacobT14]

Is that reliant on getting prisma setup? I would rather not have to depend on a 3rd party service when I don't need to... (Have no idea how prisma works and would prefer not to have to jump through a bunch of hoops if I don't need to)

[JacobT14]

Still haven't found a way around this... Even just using the in memory cache isn't working. When I issue a query I do get the cache-control extension, but it isn't caching at all. Also looking at apollo-engine it says that no cache control policy was found for that query... I'm at a loss as to how to get response caching from apollo server currently.

[tab00]

A MySQL Data Source class would be good.

[sbrichardson]

This example uses Sequelize, but it doesn't include caching, it's a more basic example.

I'm referring to the datasource cache. If you create an engine config, and define stores there it should cache automatically if you define default maxAge/cacheControl etc ?

[tab00]

Can we assume that by using Data Sources caching is handled automatically? The introductory paragraph of that page states it has "built-in support for caching, deduplication".

If you look at the Refactor using data sources commit of that example you can see dataloader (which handled caching and batching) had been removed.

[sbrichardson]

Currently, caching isn't handled automatically, unless you instantiate a datasource class with an initialize method (see below). The RestDataSource that Apollo provides does cache automatically, since they built the HTTPCache.

For a db cache you probably can start with the HTTPCache as a boilerplate (from ApolloServer source), but it needs a bit of work to customize properly.

initialize(config: DataSourceConfig<TContext>): void {
  this.context = config.context;
  this.httpCache = new HTTPCache(config.cache);
}

See: apollo-server/packages/apollo-datasource-rest/src/RESTDataSource.ts apollo-server/packages/apollo-datasource-rest/src/HTTPCache.ts apollo-server/packages/apollo-datasource/src/index.ts

There are a few points to consider with creating a Cache config for a data source

• the cacheKey design, for the HTTPCache, it uses the full request url mostly. a db would probably use a combination of the db/collection/tenant/id etc
• The cache ttls for HTTPCache respect the cache headers from the backend rest api. you would need to set those another way since a db wouldn't set those typically.
• A few revalidation, override checks

Any staff/experts are welcome to correct any of the above notes.

[tab00]

Is the cache option in ApolloServer the solution?

const { RedisCache } = require('apollo-server-cache-redis');

const server = new ApolloServer({
  typeDefs,
  resolvers,
  cache: new RedisCache({
    host: 'redis-server',
    // Options are passed through to the Redis client
  }),
  dataSources: () => ({
    moviesAPI: new MoviesAPI(),
  }),
});

apollo-server-caching states:

Internally, Apollo Server uses the KeyValueCache interface to provide a caching store for the Data Sources. An in-memory LRU cache is used by default, and we provide connectors for Memcached/Redis backends.

I noticed that the Redis implementation uses dataloader. Could that be the reason dataloader was removed from the LikesDB example, i.e. caching is handled elsewhere?

[tab00]

Can anyone provide more clarity about caching when using Data Sources with a database?

[cvburgess]

I left some sample code on https://github.com/tgriesser/knex/issues/2787 - its a bit hacky but it shows how to make a SQL datasource using Knex

See: https://github.com/tgriesser/knex/issues/2787#issuecomment-417765925

[smeijer]

The docs mention that dataSources are an alternative to dataloader. But I cannot find anywhere how to use it, or if the cache for the Rest example is request scoped?

Does anyone have good examples by now? Are you using Apollo's cache methods? Or still using Facebooks dataloader?

[cvburgess]

@smeijer its not a replacement so much as a different approach that can be combined with DataLoader if you so wish. Apollo's REST data source uses HTTPs cache headers to cache iindividual requests and does NOT do any batching. If you would like to batch requests, you can use DataLoader with a REST data source like they show in the docs.

https://www.apollographql.com/docs/apollo-server/features/data-sources.html#What-about-DataLoader

[smeijer]

@cvburgess, I did read about the batching. And I agree that we can still use DataLoader for that part. But I'm trying to find a clean solution for the database access. (Mongo to be specific).

Due to the nesting nature of Graphql, it can happen that a single graphql request, will trigger multiple hits to the database, for the same id's. This is a caching issue that DataLoader can solve, by caching the objects based on their primary keys, on request scope. Once the request has been handled, the cache is disposed of.

I'm curious if there is a way to handle that case with an 'Apollo solution'. So not the batching perse, more the cache per request to avoid redundant db hits.

[cvburgess]

@smeijer i made a Data source with caching and batching for SQL databases, you might be able to use that as a template or example to build a wrapper for Mongo. Best of luck!

https://github.com/cvburgess/SQLDataSource

[intellix]

Am currently using both:

• https://github.com/mickhansen/graphql-sequelize
• https://github.com/mickhansen/dataloader-sequelize

Was originally using ApolloEngine full query caching with field-based caching TTLs like so:

type User implements Node @cacheControl(maxAge: 90) {
  firstName: String
  lastName: String
  balance: Float @cacheControl(maxAge: 0)
}

Since ApolloEngine was deprecated the above caching has stopped working. The extensions are there but it seems it only hits the CDN.

Was looking towards migrating towards DataSources as they're the latest thing from Apollo that reference caching.

As @tab00 says, this quote from the docs is a little confusing:

Internally, Apollo Server uses the KeyValueCache interface to provide a caching store for the Data Sources. An in-memory LRU cache is used by default, and we provide connectors for Memcached/Redis backends.

Originally made a quick wrapper around a Sequelize resolver and noticed it always hits the DB. Then I start looking at RESTDataSource and see that the cache implementation is completely manual and has nothing to do with the @cacheControl() directive at all.

I guess with ApolloEngine magically handling cache based on those directives in the past it's not clear where the magic starts and ends, especially when it says it's out of the box and the example MoviesAPI DataSource doesn't handle cache at all (I assumed there was some magic).

I think a basic findOne() datasource that uses cache manually without RESTDataSource would have cleared it up for me

[lorensr]

Another (small) example is https://github.com/GraphQLGuide/apollo-datasource-mongodb/ for MongoDB

[trevorblades]

There are a few great examples in this thread, and I'll add one more from our docs: https://www.apollographql.com/docs/tutorial/data-source/#connect-a-database

Closing this issue now, but please create a new one if this pattern is still unclear!