Not getting any data from mqtt.message.consumed.total

[anandxp]

Scenario Used : We are able to successfully publish data to an MQTT topic , But the consumer shows zero. Even if we subscribe to the same topic using tool like MQTTBox , we are not getting the data we published using vmq_mzbench

!benchDL

#######

Scenario:

pool(size = 1, worker_type = mqtt_worker):

        connect([t(host, "http://dev.broker.nucleus.cantiz.com"),
                t(port,1883),
                t(client,"MQTT-Dev"),
                t(clean_session,true),
                t(keepalive_interval,30),
                t(proto_version,4), t(reconnect_timeout,4)
                ])

        wait(1 sec)
        subscribe("nucleus-event-topic-dev", 1)

pool(size = 1000, worker_type = mqtt_worker, worker_start = poisson(1000 rps)):

        connect([t(host, "http://dev.broker.nucleus.cantiz.com"),
                t(port,1883),
                t(client,fixed_client_id("pool1", worker_id())),
                t(clean_session,true),
                t(keepalive_interval,30),
                t(proto_version,4), t(reconnect_timeout,4)
                ])

        set_signal(connect, 1)
        wait_signal(connect, 1000)
        wait(4 sec)
        loop(time = 1 min, rate = 1 rps):
            publish_to_self("nucleus-event-topic-dev", random_binary(150), 1)
        disconnect() 

[ioolkos]

@anandxp thanks! does the benchmark then fail? any vmq_mzbench logs? (you can look at them on the webinterface)

You're correct that you should see the messages arriving, even with some external subscriber. So that's weird. If only the vmq_mzbench consumer doesn't get messages, then you have to play around with wait_times, as the vmq_mzbench MQTT client is completely asynchronous.

[anandxp]

@ioolkos There were no error message in any of the executions.

Got below warning and info in logs

16:08:31.300 [warning] [Undefined] <0.32.0> lager_error_logger_h dropped 11 messages in the last second that exceeded the limit of 50 messages/sec 16:08:31.399 [info] [Undefined] <0.33.0> Application mqtt_worker started on node 'mzb_director15_0@127.0.0.1'

Tried changing waits as well , But again the same response

[ioolkos]

@anandxp OK, thanks. I'll try to test your scenario script after lunch to see what's up. Thanks for your effort and patience!

[ioolkos]

@anandxp please use the following forms of publish_to_self and subscribe in your scenario. Then it works.

subscribe("nucleus-event-topic-dev/#", 1) publish_to_self("nucleus-event-topic-dev/", random_binary(150), 1)

The publish_to_self function is a little akward. I added it to allow publishers to publish (and subscribe) to their own exclusive topic. If you want a single consumer to catch all, you have to use a '#' in your topic filter)

[anandxp]

@ioolkos

Made change as per your previous comments and i am landed in Dynamic deadlock

`

06:40:00.763 [error] [ API ] <0.2142.0> Benchmark result: FAILED Dynamic deadlock detected

06:40:00.945 [error] [ API ] <0.2106.0> Stage 'pipeline - running': failed Benchmark has failed on running with reason: {benchmark_failed,{asserts_failed,1}} Stacktrace: [{mzb_pipeline,error,2, [{file,"/home/anandxavier/mzbench/server/_build/default/deps/mzbench_api/src/mzb_pipeline.erl"}, {line,90}]}, {mzb_pipeline,'-handle_cast/2-fun-0-',6, [{file,"/home/anandxavier/mzbench/server/_build/default/deps/mzbench_api/src/mzb_pipeline.erl"}, {line,172}]}]

` I referred to the dynamic deadlock related open issue https://github.com/erlio/vmq_mzbench/issues/5 and made changes accordingly , but still leading to dynamic deadlock

Also i switched between poisson and linear , changed the wait in seconds , changed the rps , poolsize , tried many combinations but again running into deadlock.

Updating the scenario i am using now.

`#!benchDL

make_install(git = "https://github.com/erlio/vmq_mzbench.git", branch = "master")

defaults("pool_size" = 1000)

pool(size = 1, worker_type = mqtt_worker):

        connect([t(host, "dev.broker.nucleus.cantiz.com"),
                t(port,1883),
                t(client,"MQTT-Dev"),
                t(clean_session,true),
                t(username,"vernemq_attinad"),
                t(password,"attinad@123"),
                t(keepalive_interval,60),
                t(proto_version,4), t(reconnect_timeout,4)
                ])

        wait(1 sec)
        subscribe("nucleus-event-topic-dev/#", 1)

pool(size = numvar("pool_size"), worker_type = mqtt_worker, worker_start = linear(100 rps)):

        connect([t(host, "dev.broker.nucleus.cantiz.com"),
                t(port,1883),
                t(username,"vernemq_attinad"),
                t(password,"attinad@123"),
                t(client,fixed_client_id("pool1", worker_id())),
                t(clean_session,true),
                t(keepalive_interval,60),
                t(proto_version,4), t(reconnect_timeout,4)
                ])

        set_signal(connect1, 1)
        wait_signal(connect1, 5000)
        wait(10 sec)
        loop(time = 1 min, rate = 1 rps):
            publish_to_self("nucleus-event-topic-dev/", random_binary(150), 1)
        disconnect()`

[anandxp]

[ioolkos]

@anandxp the publish loops won't fire, if you wait for 5000 signals with a pool size of 1000.

[anandxp]

Thanks @ioolkos for the update Even after changing wait_signal value to 1000 (PoolSize) , Getting Deadlock

@ioolkos !! Please help me in resolving this issue with deadlock !

[ioolkos]

@anandxp can you try waiting 10 seconds before the signals?

See also: https://github.com/erlio/vmq_mzbench/issues/5#issuecomment-278266252

[anandxp]

@ioolkos Thanks for your response.

Added 10 seconds wait before signals.

I tried with a small pool size and small rps for getting some results in my broker topic.

make_install(git = "https://github.com/erlio/vmq_mzbench.git", branch = "master") defaults("pool_size" = 10)

pool(size = 1,worker_type = mqtt_worker):

        connect([t(host, "dev.broker.nucleus.cantiz.com:1883"),
                t(port,1883),
                t(client,"MQTT-Dev"),
                t(clean_session,true),
                t(username,"vernemq_attinad"),
                t(password,"attinad@123"),
                t(keepalive_interval,60),
                t(proto_version,4), t(reconnect_timeout,4)
                ])

        wait(1 sec)
        subscribe("nucleus-event-topic-dev/#", 1)

pool(size = numvar("pool_size"), worker_type = mqtt_worker, worker_start = linear(10 rps)):

        connect([t(host, "dev.broker.nucleus.cantiz.com:1883"),
                t(port,1883),
                t(username,"vernemq_attinad"),
                t(password,"attinad@123"),
                t(client,fixed_client_id("pool1", worker_id())),
                t(clean_session,true),
                t(keepalive_interval,60),
                t(proto_version,4), t(reconnect_timeout,4)
                ])

        wait(10 sec)
        set_signal(connect1, 1)
        wait_signal(connect1, 10)
        wait(10 sec)
        loop(time = 1 min, rate = 1 rps):
            publish_to_self("nucleus-event-topic-dev/", random_binary(150), 1)
        disconnect()

Now i am not getting any error / deadlock with the above code.

But while monitoring the same host & topic using MQTTBox there is nothing displayed in the topic.

Also i am coming back to the original issue : Not getting any message in mqtt.message.consumed.total

[ioolkos]

@anandxp do not add the port number to the 'host' name

[anandxp]

Thanks a lot @ioolkos

I am now able to connect and subscibe the same