pact-foundation / pact-go

Golang version of Pact. Pact is a contract testing framework for HTTP APIs and non-HTTP asynchronous messaging systems.
http://pact.io
MIT License
834 stars 104 forks source link

No feedback when running provider verification through pact-go daemon #55

Closed bmarini closed 6 years ago

bmarini commented 6 years ago

I ended up bypassing the daemon and shelling out to the ruby pact-provider-verifier directly because none of the verifier's output was being returned to me in my tests.

func verifyProvider(t *testing.T, baseURL, setupURL string, pactURLS []string) {
    args := append(pactURLS,
        "--provider-base-url", baseURL,
        "--provider-states-setup-url", setupURL,
    )
    cmd := exec.Command("pact-provider-verifier", args...)

    out, err := cmd.CombinedOutput()
    if err != nil {
        t.Errorf("Provider verification failed: %s:\n\n%s\n", err, out)
    }
}

Running the verification as described in the README, I get something similar to this:

--- FAIL: TestPactProvider (0.10s)
pact_test.go:34: Error: provider verification failed:
exit status 1
bmarini commented 6 years ago

I get that this is a temporary issue until there is a native go implementation, and provider verification can run in the test process, with no need to call out to a separate binary, but running the provider verifier through the daemon currently is pretty useless since you don't see the test output.

mefellows commented 6 years ago

Thanks, would you be open to submitting A PR to make it better?

bmarini commented 6 years ago

I'd like to help out on the native go implementation of the v2 spec, is that in the works anywhere?

bethesque commented 6 years ago

We're working on a Rust implementation which will be used by all the other implementations using FFI. Until then, we're stuck with the separate process. Your help to improve the output would be greatly appreciated.

bmarini commented 6 years ago

Ok, I looked into this a bit more and I think its a matter of getting pact-go to ask for the json formatted version of the verifier output, and then packaging that up in a usable way back into the client code. I may take a stab at it.

mefellows commented 6 years ago

That JSON formatted output is a relatively new thing. Would love a PR (I'm lost for time atm).

If you're feeling really generous, I would appreciate any help on https://github.com/pact-foundation/pact-go/issues/31. This essentially acknowledges what you are talking about above, and proposes some solutions.

The Reference is still some way away (although good progress is coming), but is definitely closer than re-implementing it in its entirety in Go, particularly when you consider v3 and v4 (draft) are on their way too.

bmarini commented 6 years ago

57 should fix this :)

mefellows commented 6 years ago

Released v0.0.10