What if we could create a GraphQL server in record time for our application or library integration tests

b1zzu commented 5 years ago

Motivation

When creating GraphQL based client applications there are cases where the backend reside in a different repository owned by a different team, or it requires a complex infrastructure, or it's a proxy to other microservices, or it's unknown because our client library is meant to work independently from the backend schema. In some of this case developing integration tests for our application could requires to: mock the methods that perform the backend request (in this case we are talking of unit tests because we are not testing the integration with the backend), or to create a simple backend that return sample data and perform basic operations (like storing data in memory) which will be executed before the tests (in this case the developer would have to create and maintain a lot of boiler-template code depending from the complexity and size of the backend).

Target

We would like to solve this problem by creating a library that could be used to create a dedicated GraphQL server with a minimum set of templating and options directly from the test framework and also to control it (start/stop/reset) from the tests.

The Graphback project offers a powerful way to generate a GraphQL server from a single GraphQL schema. By working in close contact with the Graphback team we can adapt Graphback to work at runtime, wrap it inside this library, and provide a simple yet powerful API to control the server.

How to start

We can start by targeting NodeJS tests, and afterward, we can try to target browser tests using jest or karma.

Additional futures for the library could be:

Generate and provide the client queries and mutations. The Graphback project can already generate the client queries and mutations, but we will have to wrap them in runtime.
Bulk load test data or to randomly generate them when the server starts.
Snapshot and restore the backend at any moment from the tests.

Material

What Is GraphQL? [15 min]: https://www.youtube.com/watch?v=VjXb3PRL9WI
Graphback demo [13 min]: https://www.youtube.com/watch?v=4VzNZCY_Zlk

wei-lee commented 5 years ago

@b1zzu I am not sure if I get the motivation here, especially around why we want to generate Graphql servers dynamically during tests, instead of just running integration tests against one Graphql server that is designed for testing?

b1zzu commented 5 years ago

Hi @wei-lee, When the testing server resides in a dedicated space or directory (assuming the server is still build using graphback, because otherwise, the developer will have to manually create all queries, mutations, and resolvers), each time the developer wants to update the graphback schema he/she would have to: regenerate the server source code graphback generate, migrate the database graphback db, build and restart the server npm run develop, and then finally run the tests.

When the testing server is created dynamically, on runtime from the tests, the developer can update the graphback schema and the tests and then run everything together in a single action npm test.

Also if the testing server is controlled by the test framework it would be possible to start/stop/reset the server from the tests without interacting with a separate process.

wtrocki commented 5 years ago

The way I read it is that if this project is done - people do not need to write anything on the server-side or have any node.js experience. Servers will be created at runtime using different schemas and provide different use cases like interfaces unions etc.

I see this really useful for testing different use cases and the composition of the tests. This framework is targeting testing, but I see that this could be used in a much wider spectrum of tests and automation. For example testing performance of the data layer etc.

This can also bring some positive benefits for developers who want to have a runtime that is a black box - they simply do not want to commit generated source code to the repository - they want runtime instead of source code. We just simply frame this feature as a Unit test framework, but we can also get pretty generic runtime capabilities from this. (Separate spike here: https://github.com/aerogear/graphback/issues/333)

wei-lee commented 5 years ago

@b1zzu ah, ok, I think I understand now. So the idea here is to basically generate a mock graphql backend server for easier testing the client apps using graphback. Am i right?

b1zzu commented 5 years ago

@wei-lee yes exactly. A concrete use case for this are the offix integration tests.

wei-lee commented 5 years ago

@b1zzu understood. It just wasn't clear to me initially.

b1zzu commented 5 years ago

@b1zzu understood. It just wasn't clear to me initially.

@wei-lee no problem, it was something I missed in the initial description and thanks to your question we have clarified better the scope of the task :)

b1zzu commented 4 years ago

Closing this issue, because it has been split in smaller issues

aerogear / graphql-testx