vernemq / vmq_mzbench

An MQTT loadtest and usage scenario tool for VerneMQ and other MQTT systems.
Apache License 2.0
42 stars 44 forks source link

Subscriber never receiving message Fan out #28

Closed artandor closed 5 years ago

artandor commented 5 years ago

Hi, I'm currently benchmarking mqtt for both rabbitmq, vernemq and activemq.

I think mzbench is the most valuable tool to do this, but I struggle pretty hard. My subsciribers never receive the produced messages. I tried a lot of custom scenario, but even the official scenarios from this repo doesn't work.

Here is the scenario.

#!benchDL
#######
# Broadcast Scenario:
# 500 subscribers reading from the same topic "fixed/broadcast/topic"
# 1 publisher sending 1 msg/s to topic "fixed/broadcast/topic"
# Overall Msg rate: 500 msg/s
# Message Size: 150 random bytes
# Runtime: 5 min
#######

make_install(git = "https://github.com/erlio/vmq_mzbench.git",
             branch = "master")

defaults("pool_size" = 500)

pool(size = numvar("pool_size"),
     worker_type = mqtt_worker,
     woker_start = poisson(1000 rps)):

            connect(host = "some-ip",
                    port = 1883,
                    client = fixed_client_id("subscriber_pool1", worker_id()),
                    clean_session = true,
                    keepalive_interval = 60,
                    proto_version = 4,
                    reconnect_timeout = 4
                    )

            set_signal(subscribe1, 1)
            wait_signal(subscribe1, numvar("pool_size"))
            wait(10 sec)
            subscribe("fixed/broadcast/topic", 0)

pool(size = 1,
     worker_type = mqtt_worker):

            connect(host = "some-ip",
                    port = 1883,
                    client = fixed_client_id("publisher_pool1", worker_id()),
                    clean_session = true,
                    keepalive_interval = 60,
                    proto_version = 4,
                    reconnect_timeout = 4
                    )

            set_signal(connect1, 1)
            wait_signal(connect1, 1)
            wait(4 sec)
            loop(time = 5 min, rate = 1 rps):
                publish("fixed/broadcast/topic", random_binary(150), 0)
            disconnect()

Thanks in advance for answering, this is a pretty urgent benchmark :)

Have a nice day

ioolkos commented 5 years ago

Hi @artandor thanks for the issue. Can you tell me what the MZbench WebGUI tells you about the error you see?

In other words: does the test script start running, or not? do you see latency graph for the QoS 0 metrics?

artandor commented 5 years ago

The job will run, publishing will work well, but subscribing won't do anything.

I took screenshots of the graphs.

Here is the screen for the mqtt internal stats : image

And here are the screens for most importants boards in the 1 to 1 scenario.

image

image

image

Thanks a lot for your fast answer :)

ioolkos commented 5 years ago

In the "MQTT connections pane" do you have 500 or 501 connections? (1 publisher).

ioolkos commented 5 years ago

Oh, and start playing around with the timing. Slow down the connection setup, for instance, instead of poisson 1000, set it to poisson 100. Then wait with the publisher for a little longer before you start publishing: wait(20 sec) or something. The underlying bench client is brutally asynchrounous which leads to strange timing effects possibly.

EDIT: on my laptop your script works as is.

artandor commented 5 years ago

What broker are you benching on ?

I had 501 connections image

So you recommend to set 20s wait before starting the production loop ?

ioolkos commented 5 years ago

I use this to bench VerneMQ. (what broker are you testing?)

artandor commented 5 years ago

I'm testing it to bench on rabbitmq and activemq at the moment

Edit0 : Ok so, it's working on activemq now, but not on rabbitmq. Is there vernemq specific code in your package ?

Edit1: I'm still trying to find out what's wrong with RabbitMQ specificaly, I'll post an update and close the ticket once i find out.

larshesel commented 5 years ago

Hi @artandor did you figure out what the problem was?

artandor commented 5 years ago

Hi, no i did not.