Examples are too contrived to be useful

[mefellows]

The Dog API examples are useful, however as they are so simplistic the true essence of contract testing is not encouraged, and instead we encourage "API Testing" by asserting the response body, instead of how collaborating components behave.

Ideally, we have a single test case that is used consistently through all examples (perhaps sharing the domain model and common objects). My current thinking is that the E2E example should be the starting point.

[TimothyJones]

Similarly, I think there could be some refactoring around the examples:

• They currently reference the parent folder to get pact, which is a barrier to being copied and pasted into a project (a common way to get started with a new framework)
• We don't have a clear separation between provider examples and consumer examples
• Some examples clear the pact folder each run, and some don't

plus other things I haven't noticed yet.

[rahulballal]

For the problem that pact intends to solve. It would be nice if we can take a leaf out of this project https://github.com/gothinkster/realworld

Also, it would be nice if we get the meaty examples done first

1. REST api server - REST api client
2. Working with most popular test frameworks and retire the obsolete ones. Eg. Very few people use ava and tape AFAIK.
3. Replace the examples/e2e with a full docker compose setup. It is move prevalent.

All feedback is welcome ☮️

[TimothyJones]

That's a well organised project! I like the way it's easy to understand their many repositories.

I think most of the examples will be a REST API server or client - especially when it's just the test runner that is changing - but we do have a bit of conceptual difficulty in that the message pact example is a very different way of connecting, and the end-to-end example is more of an example test infrastructure. Hmm.

I absolutely agree with focussing on the most popular test frameworks (npm-stat shows mocha and jest downloads absolutely dwarf ava - although I suppose ava uptake does seem to be growing). I don't think there's any harm in keeping uncommon frameworks around though, unless they're causing maintenance overhead.

I agree that docker-compose instead of Vagrant seems like an improvement. I'm not sure that end-2-end is the right name for that example - maybe "full pact workflow"?

[mefellows]

@rahulballal I like the idea of a meaty project like realworld, I'll look into it a bit further.

Working with most popular test frameworks and retire the obsolete ones. Eg. Very few people use ava and tape AFAIK.

Yes, we mostly tend to see Jasmine, Jest and Mocha (if issues and enhancement requests are anything to go by) and should focus on ensuring these work. That being said, I see no harm in keeping other test frameworks around. If the examples help somebody then that is another user who is able to get started quickly, and a bunch of conversations that don't need to happen on our forums.

Replace the examples/e2e with a full docker compose setup. It is move prevalent.

Agree. It should be straightforward enough.

Is this something you'd be interested in helping with?

[rahulballal]

@mefellows I think if the examples are working lets not discard it. The example around gql for example is not quite a realistic use case. We should fix or remove it.

I would be willing to help with the docker-compose setup.

[mefellows]

Thanks, was chatting with @TimothyJones this afternoon and we were thinking to use npm link to also make it easier for people to copy/paste code snippets (currently we use relative pathing so that each example is also tested as part of the build).

The example around gql for example is not quite a realistic use case

Could you elaborate? Agreed it is pretty simplistic, if that's the main gripe.

I would be willing to help with the docker-compose setup.

That would be great!

[rahulballal]

re: graphql example

For a gql query like so 👍

const GET_SCOOBY = gql`query {
  dogs(name: "scooby") {
    id
    isGoodBoy
    canFindGhosts
}
}`

In react this would be consumed declaratively like so using react-appollo or the likes

<Query query={GET_SCOOBY}>
{
  ({data}) => <DogView {...data} />
}
</Query>

I am not going to pretend that I know Angular. Here is what it may look like in Angular https://medium.com/codingthesmartway-com-blog/apollo-client-for-angular-making-use-of-graphql-8d9a571e020c

So the test subject in this case would only be the graphql query. Using something like https://hackernoon.com/extensive-graphql-testing-57e8760f1c25

The case where someone would make breaking changes in schema would be:

• In development
• Not following the graphql best practice to keep the schema backwards compliant at all times

Now, the question to ask is does pact have a place in graphql ecosystem since schema enforcement is done in both places. If using relay, the queries are compiled hence contract breakage etc is detected at compile time. I am fairly new to pact but I feel pact in gql world is a hard sell.

[mefellows]

Agree there are other patterns that would be interesting complements to other forms of testing, and these should not be discounted.

There have been questions on forums related to GraphQL, and whilst our official position is that we can support it (it's just a simple abstraction over HTTP after all), it's not our primary use case. I used it on a client, and it definitely prevented issues - yes, the schema is "strongly typed" but nothing enforces both consumer and provider to use the same schema at compile time - i.e. it's possible for compilation to pass and be a schema drift (just as you could do with swagger) - you still need a simple mechanism to share the "contract" (in this case, the schema) that can't drift as unit tests tend to do.

Not following the graphql best practice to keep the schema backwards compliant at all times

Yes, but that's also a good practice for RESTful HTTP style APIs, and yet Pact exists (i.e. People do break stuff) :)

If using relay, the queries are compiled hence contract breakage etc is detected at compile time.

I could be a bit of date with current practices - how is the GraphQL schema shared between client/server? If this can be done reliably, and linked to the versions of consumers/providers that use it then all of the information is there to make a safe release.

An alternative use case for our ecosystem is supporting sharing the GraphQL schema (contract) via the broker, and allowing providers to publish verification status to it. This enables CI tools to query it during deployment to check version compatibility.

[rahulballal]

I could be a bit of date with current practices - how is the GraphQL schema shared between client/server? The server sends an introspection schema. Using relay all your interaction code can be compiled to see if it matches the schema.

Schema drift in graphql land is usually caught at the UI testing level if any. In GQL UI and Queries/Mutations are tightly paired. You can stop your client deployment right there.

So, all I am trying to say is, REST seems to be sweet spot for pact and lets get the examples good. Then we can look at things like Graphql and gRPC and the likes.

[andreasmarkussen]

I think that both simple examples and more complex examples have their right. And yes. The dog example could be exapanded.