Closed jiegillet closed 1 year ago
At the moment this is very much set up to work with the DndCharacter exercise and tests, which is a particularly tricky situation, with the random / generators.
Do you think we will want to use fuzz testing more generally, and if so, do you think the messages will still be ergonomic in those cases? The messages for the DndCharacters tests are pretty complex. I'm a bit undecided as to whether this is the right thing or not, but maybe it is, as the situation is complex and it doesn't make sense to hide it, and it teaches students about what to expect when testing random / generators. However I think in more straightforward cases, the error messages could / should be clear and simple, even if we use fuzz tests.
I agree, the dnd-character exercise is tricky because I'm using fuzz tests on random generators, but in general I think we absolutely could use fuzz tests for other exercises. For example in wordy
we have tests like
test "multiplication" <|
\() ->
Expect.equal (Just -75) <| answer "What is -3 multiplied by 25?"
We could absolutely do
, fuzz2 Fuzz.int Fuzz.int "should be able to multiply arbitrary integers" <|
\a b ->
[ "What is ", String.fromInt a, " multiplied by ", String.fromInt b, "?" ]
|> String.concat
|> answer
|> Expect.equal (Just (a * b))
and in that case, if I used + rather than * in the implementation, I would get
↓ Wordy
✗ should be able to multiply arbitrary integers
Given (0,1)
Just 1
╷
│ Expect.equal
╵
Just 0
We could also engineer better messages with fail
if we wanted.
This is the sister PR to this one, where I'm trying to to use
fuzz
for the first time.This PR makes it possible for the test runner to parse
fuzz
,fuzz2
,fuzz3
andfuzzWith
. For example, given the testWill be parsed and shown to the students as
Note that for normal tests we only show the inside of the lambda, but for fuzz tests we need to show the full thing, because the arguments are required to understand the test (otherwise this example would only render
Expect.pass
).For a wrong solution, the error message the student will see is
Note that the diagrams don't appear in this version of the messages.