Update QuickCheck test cases for qc wrapper 2.0 and eqc 1.33

[tatsuya6502]

There are number of problems in current QuickCheck test cases.

• When I replaced QuickCheck wrapper gmt_eqc with qc, I overlooked that some test cases were calling gmt_eqc:module/2 instead of qc_statem:qc_run/3 so these tests cases are still executed by gmt_eqc.
• When I made dev branch compatible with Erlang/OTP 17, I upgraded qc to 2.0. However, I overlooked that many QuickCheck test cases weren't executed (ignored) anymore due to a behavior change in recent rebar.
  • Makefile has to be updated so that they won't be ignored anymore.
  • qc 2.0 has some breaking changes in callback functions, and test cases have to be updated.
• I recently upgraded eqc from 1.26 to 1.33. It has a couple of breaking changes. qc wrapper and test cases have to be updated.

Version Info

• Hibari, dev branch (pre v0.1.11)
• Erlang/OTP 17.4
• eqc 1.33.3
• qc 2.0
• rebar 2.5.1 17 20141014_120220 (git 2.5.1-17-gad3328b)

[tatsuya6502]

Just a note; at the last release Hibari v0.1.10 (with OTP R15, eqc 1.26 and qc 1.x), the following QuickCheck properties were already failing.

• #4 QuickCheck squorum_eqc_tests:prop_squorum1 fails with quorum_error
• #5 QuickCheck admin_eqc_tests:prop_0 fails on brick_add command

[tatsuya6502]

I recently upgraded eqc from 1.26 to 1.33. It has a couple of breaking changes. qc wrapper and test cases have to be updated.

I addressed this one via the following commits:

• gdss-admin https://github.com/hibari/gdss-admin/commit/4e757a5f5f68659735442b3aab3a45187d58a948
• qc https://github.com/hibari/qc/commit/2c847bca43d60f12ea37f6a6ce6ef69628c1cc20

[tatsuya6502]

Makefile has to be updated so that they won't be ignored anymore.

I updated Makefile last week via the following commits.

• https://github.com/hibari/hibari/commit/69951141a0056261bf9a4f1ddb83f014aa5b461e
• https://github.com/hibari/hibari/commit/2fa58862702b2e4f453dce8cca6127697af2be9d

[tatsuya6502]

qc 2.0 has some breaking changes in callback functions, and test cases have to be updated.

Fixed many of them via this commit: https://github.com/hibari/gdss-admin/commit/a0a9f8c76313dd4fd45570f4608713bd08cf3af6

The following properties are still failing:

• hlog_blackbox_eqc_tests.erl:prop_commands (new)
• squorum_eqc_tests.erl:prop_squorum (known issue: #4)
• admin_eqc_tests.erl:prop_0 (known issue: #5)

[tatsuya6502]

The following properties are still failing:

• hlog_blackbox_eqc_tests.erl:prop_commands (new)

Fixed this via https://github.com/hibari/gdss-admin/commit/ba9cb39feecdd640f8e30d5ed45dd844c9abf31d Now it passes 3,000 tests.

Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:64] [hipe] [kernel-poll:true]

Eshell V6.3  (abort with ^G)
(hibariclient@127.0.0.1)1> eqc:quickcheck(eqc:numtests(3000, hlog_blackbox_eqc_tests:prop_commands())).
...
Starting Quviq QuickCheck version 1.33.3
   (compiled at {{2015,2,9},{14,18,24}})
...

OK, passed 3000 tests

100.00% {1,attempts}

52.309% {restart,1,ok}
8.024% {fold,4,ok}
7.965% {write_hunk,7,ok}
7.820% {advance_seqnum,2,ok}
3.984% {stop,1,ok}
3.953% {kill,1,ok}
3.819% {move_seq_to_longterm,3,ok}
2.003% {get_current_seqnum_and_file_position,1,ok}
1.993% {get_current_seqnum,1,ok}
1.986% {get_all_seqnums,1,ok}
1.967% {read_hunk,5,ok}
1.965% {read_hunk,4,ok}
1.933% {get_current_seqnum_and_offset,1,ok}
0.280% {advance_seqnum,2,{error,badarg}}
true
(hibariclient@127.0.0.1)2> 

[tatsuya6502]

The following properties are still failing:

squorum_eqc_tests.erl:prop_squorum (known issue: #4) admin_eqc_tests.erl:prop_0 (known issue: #5)

I'll address them in these GitHub issues after Hibari v0.1.11 release. Closing this issue.