Open ocharles opened 7 years ago
Actually, State { parts :: [Var [String] var] }
isn't just ugly, it's equally unusable:
fetchPart =
Command
{ commandGen = \(State knownParts) ->
case knownParts of
[] -> Nothing
_ -> Just (FetchPart <$> Gen.element knownParts)
Would generate Var [String] Symbolic
, but we want Var String Symbolic
. Our only option is to take a random String
from within the command execution, but then all of hedgehog's output will be misleading.
Hey sorry for the significantly delayed reply, I've been pondering this for a little while and I the best I could come up with was to introduce some commands which don't actually contact the external service, but which can be used to interrogate the Var [String] var
.
So imagine if you had something like:
-- Read :: Int -> [String] -> String
data Read v =
Read Int (Var [String] v)
You'd be able to implement execute easily because that has access to the concrete [String]
. For pre and post conditions you probably need a Map (Var [String] v) [String]
or something like that, but it's kind of unclear how this might work because you don't really have access to add/remove parts, so I don't even know what the pre/post conditions would be. There's an example in icicle which works a little bit like that.
I understand this is a bit clunky, but maybe we can iterate on the core mechanics of the idea and produce something a bit nicer.
You'd be able to implement execute easily because that has access to the concrete [String].
This seems to be my second comment, but that seems bad, because as I mentioned:
[This] would generate Var [String] Symbolic, but we want Var String Symbolic. Our only option is to take a random String from within the command execution, but then all of hedgehog's output will be misleading.
My command generator doesn't have access to the [String], so I can't take an element or calculate the length. This is presumably why you mentioned
For pre and post conditions you probably need a Map (Var [String] v) [String] or something like that
But where do the elements ([String]
) of this map come from?
I understand this is a bit clunky, but maybe we can iterate on the core mechanics of the idea and produce something a bit nicer.
Certainly, I think iterating on a concrete example might be easier. Essentially, the problem comes down to:
Shall I try and throw together a standalone example?
Yeah that sounds great, I think part of the problem here is stemming from the fact that we don't have control over the results in the system under test. If we could add parts to the system or knew what they should be then we'd be able to populate our model (i.e. the [String]
).
https://github.com/ocharles/hedgehog-scenarios/tree/master/database-constraints is essentially one of the simplest problems that I've got so far.
My external system here has some constraints, but generating commands that respects them isn't possible (in the way I've setup State
in that example).
It's a bit disappointing to find this issue, and that it's still open, since I seem to have bumped into it as well -- my question on SO. I'm guessing, based on the age of it, that it isn't something that a lot of people run into :(
Ooof, this hasn't got an answer? D:
@spacekitteh Maybe not an answer, but https://github.com/hedgehogqa/haskell-hedgehog/issues/459 explains some of my current thoughts on this.
I am doing a bit more playing with the state machine testing, in the context of testing work's REST API. For the purposes of this test, we have two end-points:
/parts?q=<q>
performs a search query and returns a list of parts that match the query q./part/<identifier>
looks up the part given asidentifier
.Internally, I know that performing a search consults an external service, and adds the search results to our local database. Therefore, I can start with a blank database, perform a search, and then lookup any of the parts that were returned. I'm trying to capture this with the following state:
The
search
command should update the list of parts with the result of the API call:But this doesn't type check!
response
is aVar [String] var
, butps
is[Var String var]
. It doesn't look like I will have any hope at reconciling the two if I keepvar
polymorphic and don't have anything more thanHTraversable
at my disposal. One vague idea would be to add a codensity-like transformation toSymbolic
, so we can at least have a working functor (HFunctor
?) for bothVar a Symbolic
andVar a Concrete
, which might be a good start.Any ideas what to do? Having
State { parts :: [Var [String] var] }
might be an option, but is not at all satisfactory 😢