Use GraphQL as the base for Hypermodel instead of ActiveRecord or Add GraphQL support

[catmando]

It seem perfect: GraphQL introspection can replace the current rails Active Record specific introspection, to determine types and relationships needed for the client. Then during the render process where hyper-model currently builds a "vector" list, it simply builds a graphql request.

On the server side we can have an adapter that turns your AR models into the appropriate GraphQL schema (like hyper-model does today but in its own internal format).

But the beauty is that if you are using something else (like Mongo), or you have complex preexisting AR models, the GraphQL schema provides a nice intermediate description of your public data, AND by the way, you get the graphQL API endpoint for free if you need to access your persisted data outside of the hyperloop app.

[catmando]

seems relevant: https://github.com/rmosolgo/graphql-ruby/issues/38

[catmando]

also a lot of work done here: https://github.com/geneeblack/graphql_model_mapper

[catmando]

This would be a solution to #7

[janbiedermann]

https://netflix.github.io/falcor/ an alternative to graphql, just for studying

[barriehadfield]

@catmando @janbiedermann I think we should seriously consider client and server GraphQL API compatibility for Hyperloop V2 plans. Complying with the schema would not necessarily mean any break in our existing API, it would just pertain the client/server communications. As you say @catmando, you get to switch one server to another (though I am not sure about the bidirectional aspect). Next year we have a lot of technology choices to make, and this one will be high up there on the list.

[catmando]

So I have spent about a week researching this and talking mainly to @janbiedermann (thanks!)

I have some tentative conclusions I want to document before moving on:

1. Take a look at relay - basically relay just like hyper-model EXCEPT that instead of dynamically creating the queries during the first render cycle, the queries are provided separately. Except for that it has a lot of very similar concepts about how to cache, how to notify the "views" (i.e component instances) that some model has changed. Relay has to deal with pushed data (i.e. subscriptions) differently than HyperModel, so there are some useful ideas about determining the data to fetch which is roughly equivalent to how Hyperloop syncronizes scopes.
2. Some things in GraphQL move to Ruby/Hyperloop very well. For example, by using GraphQL introspection you can figure out the Ruby types dynamically. Our thought is to define change ApplicationRecord to inherit from HyperGraphQL::Base, which would tell how to map graphql types to ruby classes, which would then act as much as possible like current HyperModel AR classes.
3. There is little reason to actually begin using the graphQL syntax to implement the client-server API for today's hypermodel. This was an idea as part of solving #8, but it seems not too useful. However it does seem that based on the last couple of conclusions below that there is a good chance the current hypermodel internal implementation might be short lived. Therefore we should solve #8 in a short term fashion for now.
4. Relay like us clearly puts the persistence problem at the core any client framework.
5. GraphQL mutations == Operations almost exactly. However the semantics of Subscriptions have some extra power that we might want bake into Operations. I.e. you can't quite describe how Subscriptions work, by using Operations (but almost.)
6. As I read around on this topic I kept bumping into "problems with rails", and the major problems were solved with some variation of Operations. Note above comment about GraphQL mutations, which I think is another reason for popularity of graphQL (i.e. mutations == operations.)
7. When we get serious about graphQL we have a decision: Use relay (like we use react-router, and react) with a wrapper, to make it work the hyperloop paradigm, or merge with existing hyper-model. There are obvious +/-'s here. But one is not as obvious. If we were to take the "use relay as the client base" then it is possible to come back and revisit this question, where HyperModel (as we know it today) sits on top of HyperGraphQL and uses some rails graphql server to access the database. We will learn more when we implement HyperGraphQL.

8. Anybody needing to use a GraphQL endpoint in a major way is going to probably find it easier to use Relay than adopt hyperloop. Moreover for any project that is going to have a robust published API then GraphQL is the way to go. If I already have the GraphQL schema defined, then I will want to use it on the client. This means GraphQL integration someway should be a very high priority for us. There is a growing list of GraphQL endpoints: https://github.com/APIs-guru/graphql-apis.

   Actions

   • checkout relay, see if we can use any ideas especially around synchronization;
   • look at how relay walks through the underlying concepts, we may be able to steal some documentation ideas as well;
   • Focus on #8 as a short-term fix. Begin by simply recombining vectors server side into a tree structure BEFORE processing. It may be that this will give a big enough bang to not have to modify the client side code at all.
   • Revisit docs to increase focus on three fundamental things hyperloop does: enables declarative construction of components without redundant code and boiler plates, integrates server side persistence, and provides isomorphic Operations.
   • leaving this open in light of item number 7&8

[catmando]

and this just adds fuel to the fire: https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/

[barriehadfield]

@catmando and @janbiedermann thank you for your thoughtful attention to this topic. I agree with your points above Mitch - ie fix 8 now then seriously consider sticking to the GraphQL API spec for interoperable FE and BE comms. Also, we should have a good look at Relay and perhaps there is an in-between path (or perhaps we wrap a nicer DSL).

[janbiedermann]

@catmando your conclusion is also my conclusion. In addition apollo as relay alternative might be interesting. And its absolutely interesting to see for me, during this process, the advantages of HyperModel for the developer, as a developer i dont have to write queries. As apollo advertises:

write queries, not code

HyperModel could advertise:

Just use your data, why write queries?.

Thats just what i want 👍

[janbiedermann]

I would like to add, for the future of the Hyperloop backend maybe, it could look like this: Have some GraphQL Data Provider Server Side Have Something to run Server Operations and other Server things A thin something to glue that together, rack or so, to make above things available for the client.

[catmando]

Not even sure its that hard. Remember GraphQL Mutations are exactly Like Hyperloop Operations.

So the entire hyperloop transport layer can be expressed in GraphQL.

So any graphQL server (data provider) should be able to act as a hyperloop backend.

It results in this picture:

If you have an existing GraphQL endpoint, then you just write your client app in hyperloop.

If you are creating a new App then you use ActiveRecord for persistence. If you also want a public API HyperModel will include a way to publish a public graphql endpoint based on your models.

Need to play with https://aws.amazon.com/blogs/aws/introducing-amazon-appsync/ as they are creating the persistence store directly from the graphQL schema.

[catmando]

Per @janbiedermann

GraphQL: write queries not code HyperLoop: design components, not queries

Components know what data they need, so why write anything but the components?

[sfcgeorge]

Yes! Before deciding on Hyperloop I thought I'd do React + GraphQL + ???. Hyperloop is the ???. It glues everything together and writes the boilerplate for you. To me HyperMesh is the diamond of Hyperloop; getting a front end model layer and API layer generated from your backend model layer for free is amazing (you just have to write some permissions shhhh). But if Hyperloop were built on top of even more established standards / projects it would be even better. If Hyperloop could work similar to how it does today but using GraphQL as its query format then that sounds fantastic.

[janbiedermann]

currently prerendering calls into rails context to get values synchronously and fast, but with graphql endpoints anywhere, this would need to change anyway, so prerendering calling some graphql endpoints. Thats another reason apollo binds queries to components, because during prerender they walk the tree, collect the queries, call out to the endpoints in parallel, wait for all promises to be resolved and then they walk the tree again and render. with a hyper-model approach, to fetch when required, it would be one walk of the tree and a sequence of synchronous calls to endpoints, so a lot slower probably, because the expensive thing are the callouts to the graphql endpoints Example: 4 queries, 1. 100ms 2. 10ms, 3.+4. each 20ms. Parallel calls: together: 100ms, the time of the longest call Sequencial calls: 100 + 10 + 20 + 20 = 150ms

[janbiedermann]

Now on the client, hyper-component walks the tree, renders with dummy values, collects the queries, executes them, renders again. Maybe that approach should then be used for prerendering with graphql too, -> one query, 2 renders

[catmando]

Right, prerendering with graphql should work like rendering on client (i.e. completes entire render then fetches, then re-rerenders.)