proper-testing / proper

PropEr: a QuickCheck-inspired property-based testing tool for Erlang
http://proper-testing.github.io
GNU General Public License v3.0
879 stars 168 forks source link

Target server should update process dictionary #248

Closed xspirus closed 4 years ago

xspirus commented 4 years ago

During development of the Stateful Targeted Property Based Testing (#245), some issues were found during generation of targeted types. This was caused because the main PropEr process, keeps some options in its process dictionary, not updated in the Target server.

Examples

The following examples are inspired from already existing PropEr tests.

-module(targeted_pdict).

-include_lib("proper/include/proper.hrl").
-include_lib("eunit/include/eunit.hrl").

inc_temp(Inc) ->
  case get(temp) of
    undefined -> put(temp, Inc);
    X -> put(temp, X + Inc)
  end.

get_temp() ->
  get(temp).

erase_temp() ->
  erase(temp),
  ok.

prop_sized() ->
  ?FORALL_TARGETED(X, ?USERNF(?SIZED(S, S), fun (_, _) -> ?SIZED(S, S) end),
                   begin inc_temp(X), true end).

prop_constraints() ->
  HelperType = ?SUCHTHAT(X, pos_integer(), X > 42),
  ?FORALL_TARGETED(_, ?USERNF(HelperType, fun (_, _) -> HelperType end), true).

prop_parameters() ->
  UserNF = ?USERNF(?LAZY(parameter(param)),
                   fun (_, _) -> ?LAZY(parameter(param)) end),
  ?FORALL_TARGETED(X, with_parameter(param, 1, UserNF), X =:= 1).

sized1_test() ->
  Opts = [{numtests, 12}, {max_size, 4}],
  true = proper:quickcheck(prop_sized(), Opts),
  Temp = get_temp(),
  erase_temp(),
  30 = Temp.

sized2_test() ->
  Opts = [{numtests, 3}, {start_size, 4}, {max_size, 4}],
  true = proper:quickcheck(prop_sized(), Opts),
  Temp = get_temp(),
  erase_temp(),
  12 = Temp.

constraints_test() ->
  Opts = [{constraint_tries, 1}],
  proper:quickcheck(prop_constraints(), Opts).

parameters_test() ->
  Opts = [noshrink],
  proper:quickcheck(prop_parameters(), Opts).

Running

1> targeted_pdict:sized1_test().
............
OK: Passed 12 test(s).
** exception error: no match of right hand side value 120
     in function  targeted_pdict:sized1_test/0 (targeted_pdict.erl, line 40)
2> targeted_pdict:sized2_test().
...
OK: Passed 3 test(s).
** exception error: no match of right hand side value 30
     in function  targeted_pdict:sized2_test/0 (targeted_pdict.erl, line 47)
3> targeted_pdict:constraints_test().
2020-05-03T17:31:30.213913+03:00 error: ** Generic server <0.838.0> terminating , ** Last message in was [more logs] ... {'$constraint_tries',50} ... [more logs]
** exception exit: {badmatch,{error,{cant_generate,[{targeted_pdict,prop_constraints,
                                                                    0}]}}}
     in function  proper_sa:init_target/2 (/home/spirus/Documents/Thesis/proper/src/proper_sa.erl, line 123)
     in call from proper_target:handle_call/3 (/home/spirus/Documents/Thesis/proper/src/proper_target.erl, line 276)
     in call from gen_server:try_handle_call/4 (gen_server.erl, line 661)
     in call from gen_server:handle_msg/6 (gen_server.erl, line 690)
     in call from proc_lib:init_p_do_apply/3 (proc_lib.erl, line 249)
4> targeted_pdict:parameters_test().
!
Failed: After 1 test(s).
undefined
false

As far as constraints are concerned, the logs show that the number is 50 instead of 1.