maxlinc
commented
10 years ago Oh, just had an awesome idea :)
I will probably hook it up so there's three choices for recording examples:
- never: Do not record examples
- once (or "first"): record the first example
- overwrite (or "last"): update every time a new response is received
- always: record all examples
- structurally_distinct: record samples where the structure differs (different numbers of keys or key names) but not where data differs (same keys, different values)
I think "structurally_distinct" is the most interesting and should be the default behavior. Consider how it works for a service like Get Server Details where you're polling a resource as it builds.
If you record and stub with:
- never: Pacto's auto-stubbing won't be smart enough to return something that looks like a recognizable server ("foo" is not an IP address)
- once: the server will always be stuck in a building state - your tests will hang waiting for the server to become active!
- last: the server will be active immediately. Your test won't hang, but you won't test the polling logic for even one iteration, or test that it can handle the returned data when the server is not yet complete.
- always: this is similar to the default behavior for VCR, and will behave too realistically. Your stubbed service will be just as slow as your real service. You might have zero latency, but it still takes x number of requests for a server to be "built".
- structurally_distinct: that's where it gets awesome, assuming your building servers return less data than your built ones. Pacto will essentially record each state transition, so you'll get at least two states "building and built" and perhaps some intermediate ones like "private IP available"/"public IP available".
You could achieve similar results with by explicitly tracking state, which can often be linked to a field ("status" in this case) or calculated via a callback. That'd have the advantage of working if the structure doesn't change but the nature of the data does (e.g. IP address changes from "" to a real IP). However, since that's more complex I'll likely leave it for a future enhancement.
There should at least be an option for recording samples while generating contracts. Especially since JSON::Generator-based stubbing only supports json-schema draft3 (https://github.com/thoughtworks/pacto/issues/10), but generation and example-based stubbing supports draft3 or draft4.