Closed JohnFDavenport closed 7 years ago
See request for stateful behaviour here: #417
Whereas we might provide in a language DSL something similar to http://wiremock.org/docs/stateful-behaviour/ I suggest that Hoverfly itself it should expose an API to manage a KV store for management of arbitrary data structures (think of a lightweight Redis) that would provide a common 'context' for the test and middleware. This could then be generalised to support some powerful features for population of response templates from data structures accessed by middleware. That potentially would allow developers to adopt whichever approach they found most natural for the problem. We (SpectoLabs) would provide some patterns appropriate for typical problems - FSM enabled mocks in unit-testing supported by language bindings & DSL, testing a multi-domain one-page app using WebDriver, integration or end-to-end tests using a test runner such as SOAPUI and end-to-end tests. Keep it simple.
See use-case for stateful behaviour here: SpectoLabs/hoverfly-java#42
re. Good Use Case;
Consider a search API that uses a scroll id. The requests will be exactly the same but the responses different, and eventually return a "completed" response, so that the consumer knows that there is no more data to return.
This can only be achieved with Stateful Behavior, and have already implemented using Wiremock (http://wiremock.org/docs/stateful-behaviour/). However I am interested in other options such as Hoverfly but Stateful is a must have
Thanks for the use case @robbiewareham
That sort of behaviour was supported in Mirage (our previous SV product built for British Airways - we gave it back to BA as we decided it was too embedded in the BA environment to be of use generally). Mirage had logic to detect when the same request resulted in a different response. It then built a list of responses. Mirage would always play the list back in the same order. The beauty was it required no additional logic or action on the part of the tester. In contrast, I believe the Wiremock approach requires a fair bit of work as it is based on a strict FSM approach.
The Mirage approach is ridiculously simple but in 3 years I don't remember anyone ever complaining. The developers of Mirage had an approach of 'simplest possible solution that could possibly work' which allowed such changes to be delivered quickly. There was an expectation the solution would be too simple, but in that case, as in most cases, the solution stuck.
I'm not saying we would implement the same simple scheme as Mirage, but I offer it up here for discussion.
I am generating JSON files describing the stub (currently Wiremock format) automatically from a HAR file, therefore any complexity is in code rather than on the tester.
I can't see why the simpler Mirage approach wouldn't work though
@tommysitu & @benjih I'm proposing the following for stateful behavior.
"requiresState" : {
"keyX", "value",
"keyN", "value"
}
"setsState" : {
"keyX", "value",
"keyN", "value"
}
Thoughts?
A map seems right. No caching seems right. Some initial questions:
Also, so we do not forget, this does not provide a means of detecting statefulness in capture mode. Seeing the same request body elicit different response bodies would probably be enough.
Capturing statefulness should be left for now too, as it will be pretty complicated
Why do yo need a key? For requestMatcher, do you think having multiple requiresState
values would be enough:
"requiresState" : ["Applied offer code", "Added an item"]
It might be worth to consider this type of scenario:
For example, I am testing a long polling function, the response initially showed as PENDING
and after a few request, it changes to READY
. The same applies for testing some rate limiting UI logic as well.
So what I am thinking is that in addition to per-request state transition, you can also define external state transition like this:
{
"stateToggle": [{
"current: "Wrong input password"
"next": "Login disabled temporarily"
"condition": "Count > 3"
}]
}
The problem with a list is that you would keep appending to it without removing any items. So if I had a shopping flow where the states were basket-empty->basket-contains-eggs
then changing from one to the other should remove the first state, meaning I need a key which would cause it to be overridden
The second scenario seems more complicated but it could end up being built on top of what is initially implemented. However, it would involve introducing an expression language, which would be difficult to manage. That particular example is also solvable by this:
requiresState=never input password,sets state=wrong input once
requiresState=wrong input once,sets state=wrong input twice
requiresState=wrong input twice, sets state=wrong input three times
requiresState=wrong input three times,sets state=login disabled
This is pretty much the same implementation as Wiremock;
http://wiremock.org/docs/stateful-behaviour/
{
"scenarioName": "To do list",
"requiredScenarioState": "Started",
"newScenarioState": "Cancel newspaper item added",
"request": {
"method": "POST",
"url": "/todo/items",
"bodyPatterns": [
{ "contains": "Cancel newspaper subscription" }
]
},
"response": {
"status": 201
}
}
{
"scenarioName": "To do list",
"requiredScenarioState": "Cancel newspaper item added",
"request": {
"method": "GET",
"url": "/todo/items"
},
"response": {
"status": 200,
"body" : "<items><item>Buy milk</item><item>Cancel newspaper subscription</item></items>"
}
}
If 'set state' in the final request is set back to the first "requiresState", you create a mechanism that supports searches that use a scroll ids (such as Elastic Search) very well in a repeatable manner.
Thank you @robbiewareham
What you are looking for ( @mogronalol and @tommysitu ) from the examples you just posted seems to be a finite-state machine. The nice thing about them is they are simple and deterministic.
Let's do that and leave out generalised state management.
State is now in v0.14.0 and it is documented [here]()https://docs.hoverfly.io/en/latest/pages/keyconcepts/state/state.html). If there are further issues please open a new issue for the specific problem.
We have debated how to handle state in Hoverfly but have held off as we need concrete use-cases. Bear in mind that we believe middleware will support stateful behaviour but do acknowledge that we perhaps need to add better support and guiadance.
If you have opinions, one way or another please comment here (or add a reference). We are particularly looking for: