search

eclipse-mosquitto / mosquitto

Eclipse Mosquitto - An open source MQTT broker
https://mosquitto.org
Other
9.1k stars 2.4k forks source link

Add support for hook plugins #142

Closed j3r0lin closed 8 years ago

j3r0lin commented 8 years ago

Don’t subscribe to # Sometimes it is necessary to subscribe to all messages, which are transferred over the broker, for example when persisting all of them into a database. This should not be done by using a MQTT client and subscribing to the multi level wildcard. The reason is that often the subscribing client is not able to process the load of messages that is coming its way. Especially if you have a massive throughput. Our recommended solution is to implement an extension in the MQTT broker, for example the plugin system of HiveMQ allows you to hook into the behavior of HiveMQ and add a asynchronous routine to process each incoming message and persist it to a database.

according to this, can mosquitto allow developers to write plugin to add hooks(like onConnect, onPublish, etc)?

karlp commented 8 years ago

What are you quoting here? it sounds like a pretty bald faced promo for hive.

j3r0lin commented 8 years ago

yes it is. But it really has that problem if only one consumer processes all messages from many producers. The rate of receiving messages of a single client is not enough. How could I handle this?

karlp commented 8 years ago

Works for me. You can also use topic heirarchies to split load. You can also use an alternative broker that does things differently. Hive obviously is one. VerneMQ is another, I haven't followed the docs for all known brokers, as for many people, it's perfectly reasonable to implement things with a regular client.

j3r0lin commented 8 years ago

I prefer mosquitto, but I don't think use topic to split load is the best idea, well, I need to think another way . Close this after all.

ralight commented 8 years ago

This really depends on your message load. If you are trying to process a large number of messages per second then a client will struggle - but then so would mosquitto. Don't forget the broker is only single threaded at the moment so putting in hooks would just slow everything down. This is a separate problem that needs to be addressed for the authentication plugins anyway.

I presume you've already tried this and run into problems? What rate did you achieve with the client?

j3r0lin commented 8 years ago

I don't mean to process messages in the plugin, but send it to a another place like AMQP. And also this is not my only requirements, another one is I need to know when clients connected and disconnected, this is another reason I wanted for a hook plugin.