search

hibari / hibari

Hibari is a production-ready, distributed, ordered key-value, big data store. Hibari uses chain replication for strong consistency, high-availability, and durability. Hibari has excellent performance especially for read and large value operations.
Other
273 stars 30 forks source link

Update Hibari master branch (v0.1.x) for Erlang/OTP 17.x #41

Closed tatsuya6502 closed 9 years ago

tatsuya6502 commented 9 years ago

The current dev branch of Hibari (v0.3) was already updated for OTP R16 and then for OTP 17. However, master branch of Hibari (v0.1.x) is still not compatible with these OTP releases.

Update master branch to support the latest OTP release 17.4.

tatsuya6502 commented 9 years ago

I have created release-0.1.11 branch on projects (= git repositories) that require some changes for supporting OTP R16 and 17. And then I applied similar changes to issue #24 and issue #37.

I also had to disable ubf, ubf-thrift and gdss-ubf-proto as ubf version 1 is not compatible with OTP 17.

Now gmake check-package runs without any error. Note that I'm using FreeBSD 10.1-RELEASE now. I'll try to run it on a Linux distro soon.

[minibsd] /home/tatsuya/workhub/dev/hibari-master-otp17/hibari% repo status
project hibari/                                 branch release-0.1.11
project hibari-doc/                             branch master
project hibari/lib/asciiedoc/                   branch dev
project hibari/lib/cluster_info/                branch release-0.1.11
project hibari/lib/congestion_watcher/          branch release-0.1.11
project hibari/lib/edown/                       branch release-0.1.11
project hibari/lib/gdss_admin/                  branch release-0.1.11
project hibari/lib/gdss_brick/                  branch release-0.1.11
project hibari/lib/gdss_client/                 branch release-0.1.11
project hibari/lib/gmt_util/                    branch release-0.1.11
project hibari/lib/meck/                        branch dev
project hibari/lib/partition_detector/          branch release-0.1.11
project hibari/lib/riak_err/                    branch release-0.1.11
project patches/                                branch master
[minibsd] /home/tatsuya/workhub/dev/hibari-master-otp17/hibari% uname -a
FreeBSD minibsd.hibaridb.org 10.1-RELEASE-p5 FreeBSD 10.1-RELEASE-p5 #0: Tue Jan 27 08:55:07 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
[minibsd] /home/tatsuya/workhub/dev/hibari-master-otp17/hibari% erl 
Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V6.3  (abort with ^G)
1> 
User switch command
 --> q
[minibsd] /home/tatsuya/workhub/dev/hibari-master-otp17/hibari% gmake check-package
cleaning: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
./rebar clean -r
...

compiling: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
./rebar compile
...

==> rel (compile)
==> hibari (compile)
generating: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
./rebar generate
==> rel (generate)
WARN:  'generate' command does not apply to directory /usr/home/tatsuya/workhub/dev/hibari-master-otp17/hibari
packaging: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
632b0806e6e2ee0728a3998c99831cd4 hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64.tgz
-rw-r--r--  1 tatsuya  tatsuya        85 Feb 27 08:59 hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64-md5sum.txt
-rw-r--r--  1 tatsuya  tatsuya  16847924 Feb 27 08:59 hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64.tgz
bootstrapping package: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
tar -C ./tmp -xzf ../hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64.tgz
"Invalid current directory or invalid filename mode: loader cannot read current directory\n"
{"cannot start loader",{error,invalid_current_directory}}
{"init terminating in do_boot",{error,invalid_current_directory}}
init terminating in do_boot ()
./tmp/hibari/bin/hibari start
./tmp/hibari/bin/hibari-admin bootstrap
ok
checking package: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
./tmp/hibari/bin/hibari-admin client-add hibari@127.0.0.1
ok
./tmp/hibari/bin/hibari-admin client-list
['hibari@127.0.0.1']
./tmp/hibari/bin/hibari-admin client-delete hibari@127.0.0.1
ok
./tmp/hibari/bin/hibari checkpoint
ok
./tmp/hibari/bin/hibari stop
ok
[minibsd] /home/tatsuya/workhub/dev/hibari-master-otp17/hibari%
tatsuya6502 commented 9 years ago

I pushed a couple more changes to relese-1.1.11 branch of hibari and gdss_admin projects. They are for making eunit and eqc test cases to compatible with Erlang/OTP 17 and eqc 1.33.

I ran the make target eunit and verified all eunit and eqc tests pass now under OTP 17.4. However, some eqc tests weren't executed. I believe I fixed this on develop branch one or two years ago. I'll check if I can fix this quickly.

/home/tatsuya/workhub/dev/hibari-master-otp17/hibari% uname -a
FreeBSD minibsd.hibaridb.org 10.1-RELEASE-p6 FreeBSD 10.1-RELEASE-p6 #0: Tue Feb 24 19:00:21 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

/home/tatsuya/workhub/dev/hibari-master-otp17/hibari% erl
Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V6.3  (abort with ^G)
1>
User switch command
 --> q

/home/tatsuya/workhub/dev/hibari-master-otp17/hibari% gmake eunit
compiling: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
./rebar compile
==> gmt_util (compile)
==> riak_err (compile)
==> cluster_info (compile)
==> partition_detector (compile)
==> congestion_watcher (compile)
==> meck (compile)
==> edown (compile)
==> asciiedoc (compile)
==> gdss_brick (compile)
==> gdss_client (compile)
==> gdss_admin (compile)
==> rel (compile)
==> hibari (compile)
eunit test compiling: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
./rebar eunit -r compile_only=true skip_apps='meck'
...
eunit testing: hibari-0.1.11-dev-x86_64-unknown-freebsd10.0-64 ...
./rebar eunit -r skip_apps=meck
==> gmt_util (eunit)
======================== EUnit ========================
...
gmt_util: prune_term_test (module 'gmt_util')...[0.008 s] ok
module 'gmt_time'
  module 'gmt_time_tests'
    gmt_time_tests: start_2_000_test...[0.006 s] ok
    gmt_time_tests: start_3_000_test...[0.006 s] ok
    gmt_time_tests: start_4_000_test...ok
    [done in 0.021 s]
  [done in 0.021 s]
...
module 'gmt_charset'
  module 'gmt_charset_tests'
    gmt_charset_tests: start_1_000_test...ok
    gmt_charset_tests: start_2_000_test...ok
    gmt_charset_tests: start_3_000_test...ok
    gmt_charset_tests: start_4_000_test...ok
    gmt_charset_tests: start_5_000_test...ok
    gmt_charset_tests: start_6_000_test...ok
    gmt_charset_tests: start_7_000_test...ok
    gmt_charset_tests: start_8_000_test...ok
    [done in 0.023 s]
  [done in 0.023 s]
module 'gmt_util_eqc_tests'
  gmt_eqc: eunit_module...prop_left_pad: (x10)(x100)
OK, passed 3000 tests
prop_right_pad: (x10)(x100)
OK, passed 3000 tests
prop_list_unique_u: (x10)(x100)
OK, passed 3000 tests
[1.343 s] ok
  [done in 1.346 s]
module 'gmt_time_eqc_tests'
  gmt_eqc: eunit_module...prop_time_t: (x10)(x100)
OK, passed 3000 tests
[0.424 s] ok
  [done in 0.427 s]
=======================================================
  All 14 tests passed.

...

==> gdss_admin (eunit)
======================== EUnit ========================
...
module 'brick_basic_tests'
  brick_basic_tests:43: eunit_brick_squorum_t0_test_...ok
  brick_basic_tests:48: eunit_chain_t35_test_...ok
  ...
  brick_basic_tests:58: eunit_chain_all_test_...ok
  brick_basic_tests:63: eunit_brick_itimer_test_...ok
  [done in 54.158 s]
module 'brick_txn_tests'
  ...
  brick_txn_tests:30: all_tests_test_...[0.007 s] ok
  brick_txn_tests:31: all_tests_test_...[0.014 s] ok
  brick_txn_tests:32: all_tests_test_...[0.021 s] ok
  brick_txn_tests:33: all_tests_test_...[0.028 s] ok
  brick_txn_tests:34: all_tests_test_...[0.033 s] ok
  brick_txn_tests:35: all_tests_test_...[0.022 s] ok
  brick_txn_tests:36: all_tests_test_...[0.014 s] ok
  brick_txn_tests:37: all_tests_test_...[0.021 s] ok
  brick_txn_tests:38: all_tests_test_...[0.029 s] ok
  brick_txn_tests:39: all_tests_test_...[0.029 s] ok
  ...
  [done in 0.245 s]
module 'brick_ets_tests'
module 'simple_update_counter_eqc_tests'
module 'squorum_eqc_tests'
module 'hlog_eqc_tests'
module 'hash_eqc_tests'
module 'admin_eqc_tests'
module 'simple_eqc_tests'
module 'hlog_blackbox_eqc_tests'
=======================================================
  All 15 tests passed.
==> rel (eunit)
==> hibari (eunit)
tatsuya6502 commented 9 years ago

I pushed a couple more changes to relese-1.1.11 branch of hibari and gdss_admin projects. They are for making eunit and eqc test cases to compatible with Erlang/OTP 17 and eqc 1.33.

I ran the make target eunit and verified all eunit and eqc tests pass now under OTP 17.4. However, some eqc tests weren't executed. I believe I fixed this on develop branch one or two years ago. I'll check if I can fix this quickly.

To address this problem, I filed the following GitHub issue under gdss-admin project. [gdss-admin >> #7 - Update QuickCheck test cases for qc wrapper 2.0 and eqc 1.33](Update QuickCheck test cases for qc wrapper 2.0 and eqc 1.33)

tatsuya6502 commented 9 years ago

I'm closing this task. I ran basho_bench for 6 hours and found no problem.