The decentralized indexing capabilities should be a standalone, non-opinionated module

[fubhy]

Do you want to request a feature or report a bug? Architectural proposal

What is the current behavior? The project consists of potentially independent but tightly coupled components with the end result of a graphql interface on top of a decentralized index.

What is the expected behavior? The most significant and powerful piece of this software, in my opinion, is the decentralized indexing capability.

If I could make a wish, I would want the indexing to be a completely standalone module. Instead of directly coupling it with GraphQL, the binding to the schema should be achieved through a separate adapter in the form of an API for use in custom graphql field level resolvers.

(Albeit potentially far more complex, it might even be useful in certain scenarios to decoupled the eventsourced indexing from the storage layer to allow for custom storage adapters.)

The flexibility of being able to define custom code driven field resolvers for a schema is a major part of what makes GraphQL so powerful. GraphQL is meant to be completely independent of any underlying technology (storage layer, etc.). By tightly coupling it to the index, and only the index, you are essentially eliminating a large part of this flexibility. I, as a user, want to be able to potentially further process data returned from the index, combine it with provided, dynamic arguments of the resolver and design the schema completely separate of the index and the smart contracts that I am trying to cover in the index. I might even want to build a service that is not fully decentralized and communicate with external APIs (e.g. exchanges) within the resolvers. Or I might want to implement custom mutations that read from the index or another data source to complete the required values necessary for an intended transaction.

By making the index a standalone module it would also be possible to craft platform and environment independent accessors, even outside of graphql (e.g. a JSONAPI consumer or a plain REST API).

I am personally thinking about a use-case where I want to run the graphql server within the main process of my Electron dApp and establish communication between the render process and the graphql "server" through the Electron IPC channel. This is actually what our dApp does currently. It would be my dream to then be able to simply query the decentralized index from my local graphql schema directly.

GraphQL schemas should be designed with the interface/views of the app in mind. In the end, it should be the frontend engineer who lays out the structure of the schema and it;s then the job of the backend engineer to fulfill the contract. This is another aspect of what makes GraphQL so powerful. GraphQL is more than just an accessor for a storage layer. I want to be able to combine multiple different data sources and APIs in my schema and I want to have full control of my resolver code.

For instance, the Electron dApp that I previously mentioned also uses the GraphQL schema to load the account information from a keytar'ed Wallet from the main process which reads it from the file system.

Sorry for the lengthy post. I focused a lot on graphql in this comment but my original argument is probably the most important for this request: The indexing capabilities of what you are building here are so immensely useful and powerful that it would be a shame if they would end up exclusively bound to a custom, non-extendable and rigid graphql implementation.

[leoyvens]

Those are a lot of interesting thoughts, as a quick note, we'll likely be separating the indexing and querying binaries soon, but making them un-opinionated towards each other is a lot more design work.

[Zerim]

@fubhy Thanks for the lengthy and thoughtful feedback. I think you're right on the money here.

As I mentioned in #544 in the future we aim to support multiple query languages aside from GraphQL. In that scenario it's likely that the protocol-defined interface to Indexing Nodes would be an RPC interface as you mentioned, which will provide lower level access to the indexed data.

The actual query parsing, planning and execution, would be in what we're currently calling a "Gateway Node" (subject to change), which would access the decentralized network of Indexing Nodes. The workings of the Gateway Node are "extra-protocol" meaning that in userland I would expect to see a variety of implementations that support different query languages, node discovery and node selection algorithms, resolver logic in the case of GraphQL etc. Our GraphQL Gateway Node would be just one such implementation.

Would be happy to hop on a video call, as you suggested in the other issue, to get your feedback on our latest thinking here. Feel free to shoot me an email at brandon at thegraph dot com.

[leoyvens]

We are committed to graphql for the foreseeable future and this is now old and out of context.