search

pact-foundation / pact-specification

Describes the pact format and verification specifications
MIT License
295 stars 28 forks source link

Loose checking of HTTP status range #68

Closed broeser closed 2 years ago

broeser commented 4 years ago

Software versions

Expected behaviour

As the documentation states, "You want to be as loose as possible with the matching for the response […]" (https://docs.pact.io/getting_started/matching)

Is there a way to loosely match HTTP status codes, e.g. match on 2xx ? In our case it doesn't matter much for the consumers whether it's OK, ACCEPTED or NO CONTENT, however it matters a lot whether e.g. 2xx or 4xx is returned.

Actual behaviour

We could only find exact matching on a certain status code or no matching at all.

Notes

I hope this is the right project to log this, or is this a pact specification issue? Our company is pretty new to pact.

mefellows commented 4 years ago

Hi! Thanks for raising. It's probably best raised on the specification repo, but the answer is likely to be no. It's one of the few areas we don't follow that rule.

I might leave this open though to put some commentary up on the FAQ section on docs.pact.io.

How big of an issue is it for you?

broeser commented 4 years ago

Hi! Thanks for your answer. I think clarifying in the FAQ would be good, yes. It's not a big issue for us, just was a bit unexpected.

TimothyJones commented 4 years ago

I would love this feature. It's especially useful for 4XX and 500 errors - usually I end up writing one test that I expect to do some specific error (eg 400) to cover all, but I don't actually mind what the exact code responded from the provider is- and I'd like to cover them all (but don't, because it seems like unnecessary overhead).

TimothyJones commented 4 years ago

The 200, 201 and 204 all being the same for some clients is something that I hadn't thought of, but I think is another good use case.

TimothyJones commented 4 years ago

As Matt points out, HTTP codes are a bit different to "look, any string is ok here".

Maybe we could / should do this at the DSL layer? If you're saying "my client doesn't care which of these status codes gets returned", then you'd want fuzzy matching at the provider, but it might make sense for the test to be run (on the consumer side) with each allowed status code.

bethesque commented 4 years ago

I'd be fine with adding this in the next-to-be-released spec version. Best to raise an issue in the pact-foundation/pact-specification repo.

mefellows commented 4 years ago

Not for the first time, I stand corrected. I had a feeling I'd had this exact discussion previously and the opposite result was had. This seems a good result to me, I'll move the issue.