Add Fuzz.listLengthRange or Fuzz.listMinLength

[hsavit1]

Original title: Create random fuzz list of minimum length

Is there an accepted way to do this?

[mgold]

None that I know of. This would either need to be a separate function, or a motivating use case for adding Fuzz.filter.

[zkessin]

This would be very useful in a number of cases

[rtfeldman]

@zkessin Can you give some specific code examples?

[mgold]

If we do add this, I think the best interface would be Fuzz.listLengthRange : Int -> Int -> Fuzzer a -> Fuzzer (List a) If the two ints are equal you get lists of exactly that length. If a > b then I guess you get constant []? Or just switch them?

Alternatively, if we think people will always use a large number for the second parameter, Fuzz.listMinLength : In -> Fuzzer a -> Fuzzer (List a) might make more sense.

[hsavit1]

@mgold - I think the Fuzz.listLengthRange : Int -> Int -> Fuzzer a -> Fuzzer (List a) function looks perfect

[rtfeldman]

@hsavit1 What's your intended use case for this?

I want to be very careful not to add things to the API without strong motivating use cases. Otherwise the API will get huge over time.

[gyzerok]

@rtfeldman maybe I have a use case for it. However I don't know if it's strong. How do you measure strongness?

Introduction

I was developing small pomodoro timer application in order to train myself with idea of combining property-based testing with avoiding invalid states through the type system. Here I'll try to cut things a bit, but you can easily follow link to see the full version (which is not that big).

The problem

Two pieces of the state of this application should be represented as positive numbers. These are number of already achieved pomodoros and remaining seconds during the countdown.

type alias Model =
    { achievedPomodoros : Int
    , timer : Timer
    }

type Timer
    = Countdown Int
    | Idle

Possible solutions

Lets take a look at the value within countdown.

Using opaque type

One of the solutions would be to create some opaque type PositiveInt and define some operations for it.

type PositiveInt = PositiveInt Int

fromInt : Int -> Maybe PositiveInt
add : PositiveInt -> PositiveInt -> PositiveInt
subtract : PositiveInt -> PositiveInt -> Maybe PositiveInt

Even though it gonna solve the problem from my point of view this is a total overkill in complexity.

Using property-based test

Lets say we have a following list of messages.

type Msg = Started | Stopped | OneSecondPassed

We can say that our app have following property: after timer was started it must be stopped after no more steps (except Started) than seconds in the countdown. In other words this mean that our countdown can't go below zero.

So we need to start timer and generate list with more than zero number of Stopped and OneSecondPassed messages and then fold this list using our update function.

Conclusion

While testing transitions between our Model states or in other words update function, we need to generate some list(s) of messages. One one hand generating empty list make no sense, on the other hand we may be interested in explicitly having non-empty list to be able to define some properties.

[mgold]

So, to summarize... you want to generate a list of Msgs to test that update works correctly, but need to avoid the empty list?

I don't think this is a convincing argument because I think there are better ways to test this function. First, I would write unit tests for the simple cases of "stopping a stopped timer keeps it stopped" and similar. Second, I would use map over Fuzz.intRange 1 100 or so to ensure that Countdown n is, after Started :: List.repeat (n+1) OneSecondPassed is Idle.

Finally, I think you should represent a timer as a running boolean and a remaining int, rather than a union type. That way you can pause the timer without losing the time. If you wanted to ensure that remaining was always nonnegative, you could generate a list (of any length) of Msgs and then expect that to be true after running them.

[rtfeldman]

@gyzerok Thanks for the detailed write-up!

If I understand it right, though, generating the occasional empty list is not going to break anything -it's merely guaranteed to pass whenever it comes up.

That doesn't seem like a big deal to me, considering it's pretty much innate to fuzzing that a lot of the randomly generated inputs won't ever yield interesting outputs.

[gyzerok]

@mgold thanks for your suggestions, gonna try them. And I need to use union type because there is no way to pause timer from requirements: you either finish the whole time or drop it if you was interrupted.

@rtfeldman with my particular approach empty list breaks everything, however with @mgold suggestions I can probably just do it another way.

One more thing that I found strange is lack of constant function in shrinkers. So I had to use lazy-list itself. In my case list as a whole is value in terms of shrinkers.

[hsavit1]

I apologize for not being able to come up with an example use case - when I came across this issue I was doing one of the "99 Problems" but in Elm. I forgot which one of the 99 problems I was doing at the time

[mgold]

One more thing that I found strange is lack of constant function in shrinkers.

You should be able to use Fuzz.constant.

[rtfeldman]

I'm going to close this and #75 for now. These both seem like the sort of thing someone will bring up if they run into a specific case where they would help out, and I think revisiting with that context will be more useful than leaving them open indefinitely waiting for that.

[ervasive]

I'm currently going through this series of articles but in Elm, and in one of them the author has such an example, more specifically here, starting at:

Longer win sequences

I'm not sure if this is the best example, but I though I'll share it.

Off-topic: There is a great example of "Make impossible state impossible" so I would recommend reading it to everybody interested.

[mgold]

I'm not convinced that's a good motivating example (it has a contrived feeling typical of Haskell), but supposing it is, you can:

• \list -> List.length < 4 || not (isPoints list) |> Expect.true "got points from more than 4 items"
• Use Fuzz.conditional (#91, not yet released I think)
• Use fuzz4 to get exactly four items and make a list from that

None of these are perfect but they handle the case adequately, until we get more detail on motivating uses.

(Re: contrived feeling of Haskell, I value "I'm working on a webapp and need to test x" far higher than anything written for a guide.)

[mrbackend]

See #111