Use of Batching Queries for options.separate

[joewood]

Just out of interest, when separate is specified for associations, could the resulting subsequent queries be batched using a `WHERE xxx IN ('yyy','yyy') - rather than performing separate queries? Is this a potential optimization for the future or is the advantage limited?

[mickhansen]

I'm looking at building a plugin for Sequelize that would generally convert all queries to batch friendly lookups, combine it with facebook/dataloader.

Would make it simpler to just have all graphql queries be recursive lookups and rely on batching for performance. Would also make caching trivial.

[joewood]

That's a great idea. Dataloader would be a great way to manage the caching and batching.

[juhaelee]

@mickhansen Would this implementation of dataloader with sequelize work for batching? https://github.com/scrollback/pure-graph/blob/master/src/db/helpers/createEntityLoader.js

[mickhansen]

@juhaelee That's the simple part, the hard part is making all other queries reduce to ids :)

[ofbeaton]

Is there a public example out there of someone using sequelize, graphql, and dataloader together? All examples I can find just use graphql to front existing REST API, or the SQLite example in dataloader. This project is wonderful but without dataloader it's hardly production ready.

[mickhansen]

@ofbeaton How is it less production ready than a REST API? We're using it in production with great success - You have to be carefull of too many queries but the preload generator in graphql-sequelize does a pretty good job - But yes, dataloader/.separate will make it even better.

[ofbeaton]

Until we manage to integrate dataloader, our graphql proof of concept is currently fetching multiple copies of the same queries for a single request. In our case, that makes it less production ready than a REST API.

That's why I'm keen to have it integrate with dataloader, but we have little experience with javascript, so it is slow going to get familiar and comfortable with this stuff.

That's why we liked graphql-sequelize so much. We hope to integrate dataloader before seriously considering making the transition.

[Naoto-Ida]

Is there any progress on this? My understanding is that if you want information on yourself, your friends, then your friends friends, and they all are defined as a UserType or something referencing the same table, etc., that they will be grouped and executed instead of in multiple queries for the same user id, etc. ?

[mickhansen]

@Naoto-Ida Yes and no, it depends on the level. Information on yourself and your friends (assuming friends doesn't first require a lookup on yourself) could be batched, since friends friends is dependent on friends it would have to run in a seperate batched queries.

The goal is to have 1 query per type per hierachy level, excluding id retrieval filtering queries etc.

[mickhansen]

@ofbeaton You're not using the prefetching/join helpers?

[mickhansen]

3.0.0 will land soon which removes prefetching/includes from the project and instead introduces https://github.com/mickhansen/dataloader-sequelize