search

eclipse / paho.mqtt.javascript

paho.mqtt.javascript
Other
1.14k stars 468 forks source link

Subscription to a lot of topics causes ping to time-out #124

Open svarogg opened 6 years ago

svarogg commented 6 years ago

I sometimes need to subscribe a client to a lot of topics (14k+ topics). Wildcards are not an option - I need the topic separation because different users have access to different subsets of data streams.

When a super-admin, that has an access to all streams of data connects to the system, first time he subscribes to all the topics his connection times out: "AMQJS0004E Ping timed out.". While this happens the user does not receive any notifications

Than a retry mechanism kicks in, causing a re-connect and a re-subscribe. After that everything works correctly.

A fix for #18 might also fix this problem, as we'll be able to subscribe to all those topics using one request.

P.S. Just in case it's relevant in any way, I'm using RabbitMQ as a broker.

jpwsutton commented 6 years ago

That's a lot of topics! A few thoughts / suggestions:

  1. Have you considered using wildcards instead of subscribing to 14,000+ topics? That should speed things up considerably in a number of ways.
  2. In the MQTTv3.1.1 spec, subscribe packets would count towards the keepalive, so sending that many subscription packets shouldn't cause the connection to timeout. Could this be an issue with RabbitMQ?
  3. Your right, #18 has been on my list for a while now, the Javascript client should allow multi-topic subscriptions, sadly I don't have the time to implement it at present.
svarogg commented 6 years ago

Thanks for the reply.

  1. For users that have access to all streams it it possible, and I am using a wildcard, however, there are users that just have permission to a shitton of streams( OK, not 14k, but 2-3k easily).
  2. Hmmm... not sure, but from what it seems this happens if the timeout time is smaller than the time it takes to get a response for all subscribe requests, seems like a client issue, but can't be sure about that.
  3. I might be able to convince my managers to give me some time to try and implement this myself. I took a peek in the code and it doesn't look too complicated. Can't promise anything though :(