how do I run pact-test for part of my tests?

[notyy]

I have many tests, some of them are pact tests. I run sbt pact-test to generate pact files, then run sbt pact-stubber to run the stub, at last, I run sbt test to run other tests that rely on the stub.

but when I run the first sbt pact-test, all tests will be run, so those depends on stub will fail.

what should I do to make things better? or am I misusing pact?

[davesmith00000]

Hi,

Before I try and answer, a couple of points:

1. If you run pact-stubber, it will run the tests for you, no need to do it twice.
2. You can run subsections of test suites but doing sbt "test-only *TestClassSpec" or sbt "test-only com.example.*" and then you could use the standalone stubber based on the results (which are uncompacted pact contracts but the stubber shouldn't care). However, if you have to do all this then I suspect that this indicates something is wrong with your testing approach.

It's a little hard to say in abstract but it sounds like you have your integration tests depending on a pact stub. As you said, this will lead to a vicious cycle where you need to run your tests to generate the pacts but you can't because they depend on the pacts being created in the first place.

Pact tests are there to test the agreed behaviour between a consumer and a client but nothing else. For instance, an integration test for what happens if the connection is refused should not be in a pact test because the consumer and provider had no opportunity to communicate.

My suggestion is that you try to keep pact and integration tests separate, and there are two ways to do that:

1. You can inject a "make the call" function that returns a canned response (http://engineering.monsanto.com/2015/07/28/avoiding-mocks/)
2. You can use a service mocking tool like WireMock.

I hope that helps, if not perhaps you could show me a dummy example that illustrates the problem?

[notyy]

yes, the failed tests are integration tests. I agree with you that pact should only test "the contract". what I am trying to do is separate pact and integration tests, thats why I am asking the question. I would try sbt "test-only *TestClassSpec" approach.

about integration test, I think your point is that integration tests should not depends on pact-stubber, do I understand correct? then what is pact-stubber used for? when should I use it?

[davesmith00000]

Consider the following scenario: You have a JavaScript front end making AJAX calls to Service A. Service A depends on, in other words is a consumer of, Services B and C. Perhaps it queries B and C and aggregates the results for instance.

How do you verify the contract between the JavaScript frontend (which also has consumer tests) and Service A?

Option 1: End to End tests Organise an end to end testing rig, perhaps on a VM or a duplicate on-demand cloud based testing setup that runs all of the services and makes sure they're all setup correctly with appropriate data sources and so on. This is how it was traditionally done and most people agree it's a pretty fragile and costly way to work.

Option 2: Isolate Service A. You have consumer contracts that were created by the JS pact tests and you want to verify them against Service A. Service A also has consumer tests for B and C so all you have to do is:

1. Run Service A's pact stubber which will pretend to be B and C so you don't need to run them or set them up;
2. Run Service A configured to talk to the host + port of the stubber;
3. Verify (sbt pact-verify) the JS pacts against Service A.

You can do all that locally, or on a CI box or wherever you like with no special tooling.

Does that help?

[notyy]

thank you for your detailed explanation. now I understand it(I think). as I am new to pact, I will do more hands on practice.

thank you again.