Long Delay on Multiple Topic Subscription

[ghost]

I recently started noticing that subscribing to multiple topics at once (multiple topics per SUBSCRIBE packet) takes a very long time. In a simple test I tried subscribing to multiple topics of the form sensor/{UUID}/# and got the following results:

[2015-10-29 13:04:07.423] [I] Subscribing 50 topics: 26ms
[2015-10-29 13:04:07.502] [I] Subscribing 100 topics: 75ms
[2015-10-29 13:04:07.834] [I] Subscribing 150 topics: 327ms
[2015-10-29 13:04:08.485] [I] Subscribing 200 topics: 646ms
[2015-10-29 13:04:09.497] [I] Subscribing 250 topics: 1010ms
[2015-10-29 13:04:11.633] [I] Subscribing 300 topics: 2126ms
[2015-10-29 13:04:15.666] [I] Subscribing 350 topics: 4029ms
[2015-10-29 13:04:22.090] [I] Subscribing 400 topics: 6421ms
[2015-10-29 13:04:32.312] [I] Subscribing 450 topics: 10218ms
[2015-10-29 13:04:46.822] [I] Subscribing 500 topics: 14503ms

I tried switching emqttd with vernemq (another MQTT broker written in Erlang) and got the following results:

[2015-10-29 13:09:29.313] [I] Subscribing 50 topics: 14ms
[2015-10-29 13:09:29.338] [I] Subscribing 100 topics: 21ms
[2015-10-29 13:09:29.366] [I] Subscribing 150 topics: 24ms
[2015-10-29 13:09:29.470] [I] Subscribing 200 topics: 101ms
[2015-10-29 13:09:29.574] [I] Subscribing 250 topics: 100ms
[2015-10-29 13:09:29.701] [I] Subscribing 300 topics: 101ms
[2015-10-29 13:09:29.889] [I] Subscribing 350 topics: 184ms
[2015-10-29 13:09:30.043] [I] Subscribing 400 topics: 145ms
[2015-10-29 13:09:30.219] [I] Subscribing 450 topics: 172ms
[2015-10-29 13:09:30.336] [I] Subscribing 500 topics: 114ms

@erylee do you have any idea about what can be causing this? Is there anything that can be made to speed things up?

[emqplus]

What’s benchmark tool you used? How many SUBSCRIBE requests per second? emqttd will create topic trees, subscriptions in mnesia database.

在 2015年10月29日，下午9:10，Bruno M. Custódio notifications@github.com 写道：

I recently started noticing that subscribing to multiple topics at once (multiple topics per SUBSCRIBE packet) takes a very long time. In a simple test I tried subscribing to multiple topics of the form sensor/{UUID}/# and got the following results:

[2015-10-29 13:04:07.423] [I] Subscribing 50 topics: 26ms [2015-10-29 13:04:07.502] [I] Subscribing 100 topics: 75ms [2015-10-29 13:04:07.834] [I] Subscribing 150 topics: 327ms [2015-10-29 13:04:08.485] [I] Subscribing 200 topics: 646ms [2015-10-29 13:04:09.497] [I] Subscribing 250 topics: 1010ms [2015-10-29 13:04:11.633] [I] Subscribing 300 topics: 2126ms [2015-10-29 13:04:15.666] [I] Subscribing 350 topics: 4029ms [2015-10-29 13:04:22.090] [I] Subscribing 400 topics: 6421ms [2015-10-29 13:04:32.312] [I] Subscribing 450 topics: 10218ms [2015-10-29 13:04:46.822] [I] Subscribing 500 topics: 14503ms I tried switching emqttd with vernemq and got the following results:

[2015-10-29 13:09:29.313] [I] Subscribing 50 topics: 14ms [2015-10-29 13:09:29.338] [I] Subscribing 100 topics: 21ms [2015-10-29 13:09:29.366] [I] Subscribing 150 topics: 24ms [2015-10-29 13:09:29.470] [I] Subscribing 200 topics: 101ms [2015-10-29 13:09:29.574] [I] Subscribing 250 topics: 100ms [2015-10-29 13:09:29.701] [I] Subscribing 300 topics: 101ms [2015-10-29 13:09:29.889] [I] Subscribing 350 topics: 184ms [2015-10-29 13:09:30.043] [I] Subscribing 400 topics: 145ms [2015-10-29 13:09:30.219] [I] Subscribing 450 topics: 172ms [2015-10-29 13:09:30.336] [I] Subscribing 500 topics: 114ms @erylee https://github.com/erylee do you have any idea about what can be causing this? Is there anything that can be made to speed things up?

— Reply to this email directly or view it on GitHub https://github.com/emqtt/emqttd/issues/365.

[ghost]

I just wrote a simple Java application that uses an MQTT client (Paho) and connects to the broker. Then, it issues 10 SUBSCRIBE packets with a variable number of topic filters inside (each of those log lines corresponds to a SUBSCRIBE request).

[emqplus]

@bmcustodio, I will benchmark this issue later:)

[emqplus]

@bmcustodio, This issue fixed in 0.14 release.

Benchmark on my Air:

80ms to subscribe 500
130ms to subscribe 1000
3s to subscribe 10000