How to run vmq_mzbench ?

[liucy2007975]

I have installed mzbench and vmq_mzbench , I start mzbench first ,then I just copy vmq_mzbench/sample.scenario to mzbench/examples/mqtt.erl , Then I excute command ' ./bin/mzbench run examples/mqtt.erl ' , but nothing happened , I got the log: 06:16:24.919 [info] [ API ] <0.253.0> Node repo: {git_install_spec, "https://github.com/machinezone/mzbench.git", "c5705ba5b73cec07522faa77a2f45d8d50ae7171", "node",[]} 06:16:24.920 [info] [ API ] <0.253.0> Stage 'pipeline - init': started 06:16:24.921 [info] [ API ] <0.257.0> Starting benchmark at 1461305784 #{benchmark_name => "mqtt.erl", cloud => undefined, deallocate_after_bench => true, director_host => undefined, emulate_bench_crash => false, env => [{<<"mzb_script_name">>, <<"mqtt.erl">>}], exclusive_node_usage => true, id => 0, initial_user => "root", log_compression => deflate, log_file => "log.txt", metric_update_interval_ms => 10000, metrics_compression => none, metricsfile => "metrics~s.txt", node_install_spec => {git_install_spec,"https://github.com/machinezone/mzbench.git", "c5705ba5b73cec07522faa77a2f45d8d50ae7171", "node", []}, node_log_port => 4801, node_management_port => 4802, nodes_arg => 1, provision_nodes => true, purpose => "bench-0-1461305784", req_host => <<"localhost:4800">>, script => #{body => <<"%% Here's a stupid, quick example to give you an idea.\n%% Paste this to the scenario field in MZBench web frontend and\n%% run it.\n\n[\n{make_install, [{rsync, \"/root/vernemq/vmq_mzbench\"},\n{exclude, \"deps\"}]},\n\n{pool, [{size, {numvar, \"publisher-number\", 10000}},\n {worker_type, mqtt_worker},\n {worker_start, {linear, {{numvar, \"publisher-rps\", 20}, rps}}}\n], \n[\n {connect, [{t, host, \"172.16.38.164\"},\n {t, port, 1883},\n {t, client, {random_client_id, 18}},\n {t, clean_session, false},\n {t, keepalive_interval, 60}]},\n{wait, {5, sec}},\n\n{loop, [\n {time, {30, sec}},\n {rate, {2, rps}}\n],\n[\n{publish, \"testtopic\", <<\"hello from vmq_mzbench\">>, 0}\n]}]},\n\n{pool, [{size, 1},\n{worker_type, mqtt_worker}], [\n{connect, [{t, host, \"172.16.38.164\"},\n\t{t, port, 1883},\n\t{t, client, \"sub2\"},\n\t{t, clean_session, true},\n\t{t, keepalive_interval, 60}\n]},\n{wait, {1, sec}},\n{subscribe, \"testtopic\", 0}\n]}\n].\n">>, filename => "E365C8742D2375E3CB10AE7B78DD4362F1D6499.erl", name => "mqtt.erl"}, vm_args => [], worker_hosts => []} 06:16:24.925 [info] [ API ] <0.253.0> Stage 'pipeline - init': finished 06:16:24.925 [info] [ API ] <0.253.0> Stage 'pipeline - checking_script': started 06:16:24.967 [info] [ API ] <0.253.0> Stage 'pipeline - checking_script': finished 06:16:24.967 [info] [ API ] <0.253.0> Stage 'pipeline - allocating_hosts': started 06:16:24.968 [info] [ API ] <0.261.0> Allocating 2 hosts in undefined cloud... 06:16:24.968 [info] [ API ] <0.261.0> Allocated hosts: [undefined] @ ["localhost"] 06:16:24.969 [info] [ API ] <0.253.0> Stage 'pipeline - allocating_hosts': finished 06:16:24.969 [info] [ API ] <0.253.0> Stage 'pipeline - provisioning': started 06:16:24.978 [info] [ API ] <0.262.0> Provisioning nodes: ["localhost"] With config: #{benchmark_name => "mqtt.erl", cloud => undefined, deallocate_after_bench => true, director_host => "localhost", emulate_bench_crash => false, env => [{<<"mzb_script_name">>,<<"mqtt.erl">>}], exclusive_node_usage => true, id => 0, initial_user => "root", log_compression => deflate, log_file => "log.txt", metric_update_interval_ms => 10000, metrics_compression => none, metricsfile => "metrics~s.txt", node_install_spec => {git_install_spec,"https://github.com/machinezone/mzbench.git", "c5705ba5b73cec07522faa77a2f45d8d50ae7171", "node",[]}, node_log_port => 4801, node_management_port => 4802, nodes_arg => 1, provision_nodes => true, purpose => "bench-0-1461305784", req_host => <<"localhost:4800">>, script => #{body => <<"%% Here's a stupid, quick example to give you an idea.\n%% Paste this to the scenario field in MZBench web frontend and\n%% run it.\n\n[\n{make_install, [{rsync, \"/root/vernemq/vmq_mzbench\"},\n{exclude, \"deps\"}]},\n\n{pool, [{size, {numvar, \"publisher-number\", 10000}},\n {worker_type, mqtt_worker},\n {worker_start, {linear, {{numvar, \"publisher-rps\", 20}, rps}}}\n], \n[\n {connect, [{t, host, \"172.16.38.164\"},\n {t, port, 1883},\n {t, client, {random_client_id, 18}},\n {t, clean_session, false},\n {t, keepalive_interval, 60}]},\n{wait, {5, sec}},\n\n{loop, [\n {time, {30, sec}},\n {rate, {2, rps}}\n],\n[\n{publish, \"testtopic\", <<\"hello from vmq_mzbench\">>, 0}\n]}]},\n\n{pool, [{size, 1},\n{worker_type, mqtt_worker}], [\n{connect, [{t, host, \"172.16.38.164\"},\n\t{t, port, 1883},\n\t{t, client, \"sub2\"},\n\t{t, clean_session, true},\n\t{t, keepalive_interval, 60}\n]},\n{wait, {1, sec}},\n{subscribe, \"testtopic\", 0}\n]}\n].\n">>, filename => "E365C8742D2375E3CB10AE7B78DD4362F1D6499.erl", name => "mqtt.erl"}, user_name => undefined, vm_args => [], worker_hosts => []} 06:16:24.979 [info] [ API ] <0.262.0> [ MKDIR ] /tmp/mz/bench-0-1461305784 06:16:24.980 [info] [ API ] <0.262.0> There's only one host, no need to make ntp check 06:16:24.980 [info] [ API ] <0.262.0> NTP check result: ok 06:16:24.988 [info] [ API ] <0.263.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;ps -ef | grep beam | grep -v grep | grep -v mzbench_api && ~/.local/share/mzbench/bin/mzbench stop; true " (<0.263.0>) 06:16:25.262 [info] [ API ] <0.263.0> [ EXEC ] OK in 273.912 ms (<0.263.0>) 06:16:25.266 [info] [ API ] <0.265.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;uname -sr " (<0.265.0>) 06:16:25.520 [info] [ API ] <0.265.0> [ EXEC ] OK in 253.692 ms (<0.265.0>) 06:16:25.521 [info] [ API ] <0.266.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;erl -noshell -eval 'io:fwrite(\"~s\", [erlang:system_info(version)]).' -s erlang halt " (<0.266.0>) 06:16:26.391 [info] [ API ] <0.266.0> [ EXEC ] OK in 870.795 ms (<0.266.0>) 06:16:26.395 [info] [ API ] <0.262.0> Missing tarballs: [{"linux-3.10.0-229.el7.x86_64_erts-7.3", "/root/.local/cache/mzbench_api/packages/node-c5705ba5b73cec07522faa77a2f45d8d50ae7171-linux-3.10.0-229.el7.x86_64_erts-7.3.tgz"}] 06:16:26.395 [info] [ API ] <0.267.0> Building package node on localhost 06:16:26.396 [info] [ API ] <0.268.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;mkdir -p /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395878 && cd /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395878 && git clone https://github.com/machinezone/mzbench.git deployment_code && cd deployment_code && git checkout c5705ba5b73cec07522faa77a2f45d8d50ae7171 && cd ./node && make generatetgz && mv .tgz /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395511.tgz " (<0.268.0>) 06:24:29.905 [info] [ API ] <0.268.0> [ EXEC ] OK in 483509.609 ms (<0.268.0>) 06:24:29.907 [info] [ API ] <0.267.0> Downloading package node from localhost 06:24:29.907 [info] [ API ] <0.267.0> [ COPY ] /root/.local/cache/mzbench_api/packages/node-c5705ba5b73cec07522faa77a2f45d8d50ae7171-linux-3.10.0-229.el7.x86_64_erts-7.3.tgz <- /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395511.tgz 06:24:29.937 [info] [ API ] <0.445.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;mkdir -p /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395576 && cd /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395576 && tar xzf /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395511.tgz " (<0.445.0>) 06:24:30.455 [info] [ API ] <0.445.0> [ EXEC ] OK in 517.022 ms (<0.445.0>) 06:24:30.455 [info] [ API ] <0.446.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;mkdir -p ~/.local/share && rsync -aW /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395576/ ~/.local/share " (<0.446.0>) 06:24:31.559 [info] [ API ] <0.446.0> [ EXEC ] OK in 1103.804 ms (<0.446.0>) 06:24:31.560 [info] [ API ] <0.447.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;rm -rf /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395511.tgz; rm -rf /tmp/bench_mzbench_api_cs_ytch-web2_1461_305786_395576; true " (<0.447.0>) 06:24:31.876 [info] [ API ] <0.447.0> [ EXEC ] OK in 316.419 ms (<0.447.0>) 06:24:31.883 [info] [ API ] <0.449.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;uname -sr " (<0.449.0>) 06:24:32.136 [info] [ API ] <0.449.0> [ EXEC ] OK in 252.766 ms (<0.449.0>) 06:24:32.136 [info] [ API ] <0.450.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;erl -noshell -eval 'io:fwrite(\"~s\", [erlang:system_info(version)]).' -s erlang halt " (<0.450.0>) 06:24:33.006 [info] [ API ] <0.450.0> [ EXEC ] OK in 869.997 ms (<0.450.0>) 06:24:33.007 [info] [ API ] <0.262.0> Missing tarballs: [{"linux-3.10.0-229.el7.x86_64_erts-7.3", "/root/.local/cache/mzbench_api/packages/vmq_mzbench-1461.306271.883036-linux-3.10.0-229.el7.x86_64_erts-7.3.tgz"}] 06:24:33.008 [info] [ API ] <0.452.0> Building package vmq_mzbench on localhost 06:24:33.008 [info] [ API ] <0.453.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;mkdir -p /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_8274 && cd /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_8274 && rsync -aW /root/vernemq/vmq_mzbench deployment_code && cd deployment_code/ && make generate_tgz && mv .tgz /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_7947.tgz " (<0.453.0>) 06:24:33.412 [error] [ API ] <0.453.0> [ EXEC ] Command execution failed in 403.987 ms Cmd: bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;mkdir -p /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_8274 && cd /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_8274 && rsync -aW /root/vernemq/vmq_mzbench deployment_code && cd deployment_code/ && make generate_tgz && mv .tgz /tmp/bench_mzbench_api_cs_ytch-web2_1461_3062737947.tgz " Exit code: 2 Output: make: ** No rule to make target `generate_tgz'. Stop.

06:24:33.413 [info] [ API ] <0.454.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;rm -rf /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_7947.tgz; rm -rf /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_8011; true " (<0.454.0>) 06:24:33.670 [info] [ API ] <0.454.0> [ EXEC ] OK in 257.413 ms (<0.454.0>) 06:24:33.671 [error] [ API ] <0.253.0> Stage 'pipeline - provisioning': failed Command returned 2: bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;mkdir -p /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_8274 && cd /tmp/bench_mzbench_api_cs_ytch-web2_1461_306273_8274 && rsync -aW /root/vernemq/vmq_mzbench deployment_code && cd deployment_code/ && make generatetgz && mv .tgz /tmp/bench_mzbench_api_cs_ytch-web2_1461_3062737947.tgz " Command output: make: ** No rule to make target `generate_tgz'. Stop.

06:24:33.672 [info] [ API ] <0.253.0> Bench final: failed 06:24:33.672 [info] [ API ] <0.253.0> Stage 'finalize - saving_bench_results': started 06:24:33.673 [info] [ API ] <0.253.0> Stage 'finalize - saving_bench_results': finished 06:24:33.673 [info] [ API ] <0.253.0> Stage 'finalize - sending_email_report': started 06:24:33.688 [info] [ API ] <0.253.0> Stage 'finalize - sending_email_report': finished 06:24:33.688 [info] [ API ] <0.253.0> Stage 'finalize - stopping_collectors': started 06:24:33.688 [info] [ API ] <0.253.0> Stage 'finalize - stopping_collectors': finished 06:24:33.688 [info] [ API ] <0.253.0> Stage 'finalize - cleaning_nodes': started 06:24:33.689 [info] [ API ] <0.459.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;cd /tmp/mz/bench-0-1461305784; ~/.local/share/mzbench/bin/mzbench stop; true " (<0.459.0>) 06:24:35.269 [info] [ API ] <0.459.0> [ EXEC ] OK in 1579.866 ms (<0.459.0>) 06:24:35.269 [info] [ API ] <0.460.0> [ EXEC ] bash -c "export PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin'; source /etc/profile;rm -rf /tmp/mz/bench-0-1461305784 " (<0.460.0>) 06:24:35.524 [info] [ API ] <0.460.0> [ EXEC ] OK in 255.056 ms (<0.460.0>) 06:24:35.525 [info] [ API ] <0.253.0> Stage 'finalize - cleaning_nodes': finished 06:24:35.525 [info] [ API ] <0.253.0> Stage 'finalize - deallocating_hosts': started 06:24:35.525 [info] [ API ] <0.461.0> Deallocator has started 06:24:35.525 [info] [ API ] <0.253.0> Stage 'finalize - deallocating_hosts': finished

how should I do to run the vmq_mzbench？

[ioolkos]

Hi there :) thanks...can you try adding a trailing slash to the rsync path in the scenario file? in your case, the path should probably be /root/vernemq/vmq_mzbench/

[liucy2007975]

Thank you for reply ! yes, I save the vmq_mzbench and mzbench in the same path /root/vernemq ,

[liucy2007975]

%% Here's a stupid, quick example to give you an idea. %% Paste this to the scenario field in MZBench web frontend and %% run it.

[ {make_install, [{rsync, "/root/vernemq/vmq_mzbench"}, {exclude, "deps"}]},

{pool, [{size, {numvar, "publisher-number", 100}}, {worker_type, mqtt_worker}, {worker_start, {linear, {{numvar, "publisher-rps", 20}, rps}}} ], [ {connect, [{t, host, "172.16.38.164"}, {t, port, 1883}, {t, client, {random_client_id, 18}}, {t, clean_session, false}, {t, keepalive_interval, 60}]}, {wait, {5, sec}},

{loop, [ {time, {30, sec}}, {rate, {2, rps}} .....

[ioolkos]

{make_install, [{rsync, "/root/vernemq/vmq_mzbench/"}, {exclude, "deps"}]},

observe the trailing slash I added here!

[liucy2007975]

Thank you very mach ! I missed the last '/' in the path . There is another Issue ,after finish the test ,the vernemq server has so many connections status in 'CLOSE_WAIT' " netstat -an | grep 'CLOSE_WAIT' | wc -l " ---> 782561 that make the new clients can hardly connect.

[ioolkos]

Very interesting, I don't see this here. Question is, if this is related to some TCP configs on OS level.