quicktheories / QuickTheories

Property based testing for Java 8
Apache License 2.0
505 stars 51 forks source link

Is there a clean way to generate values, omitting particular values? #36

Closed michaelmp closed 6 years ago

michaelmp commented 6 years ago

Using assuming tends to result in InvalidStateException if the assumption is overly constraining. The README mentions this and warns against overuse of assuming. In practice, I run into this exception frequently enough (say, 1/10 runs) that it is obnoxious.

My use case is that I want to generate all possible strings (or characters, for starters) with the exception of a handful of defined values.

I wrote this code to get a Gen<Character> that omits some defined Characters:

    /**
     * Create a {@link Gen} of {@link Gen} ranges separated by the values at
     * {@code chars}.
     *
     * @param chars
     * @return
     */
    public static Gen<Character> basicLatinCharactersOmitting(List<Character> chars) {
        List<Character> sortedChars = chars
                .stream()
                .sorted((c1, c2) -> c1.compareTo(c2))
                .collect(Collectors.toList());
        if (sortedChars.isEmpty()) {
            return Generate.characters(MIN_CODE_POINT, MAX_CODE_POINT);
        }
        List<Gen<Character>> q = new ArrayList<>();
        for (int i = 0; i < sortedChars.size(); i++) {
            Character c = sortedChars.get(i);
            int codePoint = (int) c;
            if (i == 0) {
                if (codePoint - MIN_CODE_POINT > 1) {
                    q.add(Generate.characters(MIN_CODE_POINT, codePoint - 1));
                }
            }
            if (i == sortedChars.size() - 1) {
                if (MAX_CODE_POINT - codePoint > 1) {
                    q.add(Generate.characters(codePoint + 1, MAX_CODE_POINT));
                }
            }
            if (i > 0 && i < sortedChars.size() - 1) {
                Character prior = sortedChars.get(i - 1);
                if (codePoint - prior > 1) {
                    q.add(Generate.characters(prior + 1, codePoint - 1));
                }
            }
        }
        if (q.size() == 1) {
            return q.get(0);
        } else {
            return q.stream().reduce((a, b) -> a.mix(b)).get();
        }
    }

This helper method is more complex than I'd like to have in my codebase. Is there a simpler way to achieve this using the API? Is it worth extending the API with this kind of functionality? Thanks.

hcoles commented 6 years ago

Dy default quicktheories will currently make ten attempts to generate a valid value before giving up (so assumptions shouldn't filter more than about 10% of the values).

Presumably you are filtering out somewhere near 12 of the 128 possible values?

This default can be changed globally with the QT_ATTEMPTS system property, and overridden on a per theory basic by calling.

withGenerateAttempts(xxx);

Playing with these parameters is easiest solution, but the number of attempts is not attached to the Gen, so it needs to be either changed globally or on a per usage basis.

As the domain you are working with is small an alternative approach would be to do this as a pick list of the valid values.

Something like

  public static Gen<Character> basicLatinCharactersOmitting(Set<Character> chars) {
    List<Character> allChars = <generate a list of all 128 latin characters>;
    return Generate.pick(allChars.stream().filter(chars::contains).collect(Collectors.toList()));
  }
michaelmp commented 6 years ago

Thanks Henry, I went with the exhaustive list for my tests since the range of code points is small.