Need more control over distribution of lengths of action sequences

[dcoutts]

The existing code uses an interesting distribution:

      arbActions s step = sized $ \n ->
        let w = n `div` 2 + 1
         in frequency
              [ (1, return ([], []))
              , (w, ... -- generate an action
              ]

It flips a weighted coin and decides to continue or stop. I'm rather ignorant when it comes to distributions, so I'm not sure what this distribution is called, nor do I know its properties.

What I observe however is a much higher number of empty action sequences than I would expect, or desire: at the default QC size 100, 6-8% of all runs have empty action sequences. This is a typical result (100 tests, size 100):

Actions total (100 in total):
58% NumActions "10 <= n < 100"
34% NumActions "1 <= n < 10"
 8% NumActions "n == 0"

Note also that this does not improve if one uses more tests. This is because QC cycles the size over and over. From the QC code:

    -- e.g. with maxSuccess = 250, maxSize = 100, goes like this:
    -- 0, 1, 2, ..., 99, 0, 1, 2, ..., 99, 0, 2, 4, ..., 98.

So even with lots of tests, we still end up with lots of those tests being at small sizes. With the existing method of choosing the length of the action sequence, that results in lots of short sequences.

This leads on to the general lack of control. One can of course boost the QC size for the test property. But this also boosts the size of all the actions within, which is not normally what one wants. Yes, one can compensate again by dialling down the size of the individual actions, having boosted the size overall.

What would be better is a more direct knob to turn to adjust the length of the action sequence, perhaps as a function of the size.

Suggestion: add to one of the type classes an optional method that is a generator for the length of the action list. This would be a normal QC Gen type, so would have access to the size in the usual way. The default could still be the existing distribution (though I'd be curious to hear any justification of the existing choice), but it would allow overriding for models where longer action sequences are appropriate.

[MaximilianAlgehed]

C.f. #84

One problem with longer action sequences is that QuickCheck has poor performance when individual tests have a lot of calls to tabulate (which you get when you have long action sequences) so the performance will degrade non-linearly.

[MaximilianAlgehed]

8 % empty sequences is unfortunate. I don't know who wrote that code originally, probably John, and this was probably not a deliberate choice rather a consequence of "it was faster to do it like this and there was a lot to do".