whatyouhide / stream_data

Data generation and property-based testing for Elixir. 🔮
https://hexdocs.pm/stream_data
864 stars 66 forks source link

Intermittent test failure #149

Closed bryanenders closed 1 year ago

bryanenders commented 3 years ago

The test suite fails with the seed 531973 producing

  1) test gen all supports generation and filtering clauses (ExUnitPropertiesTest)
     test/ex_unit_properties_test.exs:6
     ** (StreamData.FilterTooNarrowError) too many consecutive elements were filtered out.
      To avoid this:

       * make sure the generation space contains enough values that the chance of a generated
         value being filtered out is small. For example, don't generate all integers and filter
         out odd ones in order to have a generator of even integers (since you'd be taking out
         half the generation space).

       * keep an eye on how the generation size affects the generator being filtered. For
         example, you might be filtering out only a handful of values from the generation space,
         but small generation sizes might make the generation space much smaller hence increasing
         the probability of values that you'd filter out being generated.

       * try to restructure your generator so that instead of generating many values and taking
         out the ones you don't want, you instead generate values and turn all of them into
         values that are suitable. For example, multiply integers by two to have a generator of
         even values instead of filtering out all odd integers.

     code: check all {string, list} <- data do
     stacktrace:
       test/ex_unit_properties_test.exs:10: anonymous fn/4 in ExUnitPropertiesTest."test gen all supports generation and filtering clauses"/1
       (stream_data 0.5.0) lib/stream_data/lazy_tree.ex:67: StreamData.LazyTree.filter_map/2
       (stream_data 0.5.0) lib/stream_data.ex:380: StreamData.bind_filter/5
       (stream_data 0.5.0) lib/stream_data.ex:351: anonymous fn/5 in StreamData.bind_filter/3
       (stream_data 0.5.0) lib/stream_data.ex:202: anonymous fn/3 in StreamData.bind_filter/5
       (stream_data 0.5.0) lib/stream_data/lazy_tree.ex:36: StreamData.LazyTree.map/2
       (stream_data 0.5.0) lib/stream_data.ex:384: StreamData.bind_filter/5
       (stream_data 0.5.0) lib/stream_data.ex:351: anonymous fn/5 in StreamData.bind_filter/3
       (stream_data 0.5.0) lib/stream_data.ex:202: anonymous fn/4 in StreamData.scale/2
       (stream_data 0.5.0) lib/stream_data.ex:202: StreamData.bind_filter/5
       (stream_data 0.5.0) lib/stream_data.ex:351: anonymous fn/5 in StreamData.bind_filter/3
       (stream_data 0.5.0) lib/stream_data.ex:202: StreamData.check_all/7
       test/ex_unit_properties_test.exs:19: (test)