Middleware based connection management

[256dpi]

This is an RFC for a middleware based management of connections. This API would simplify the development of custom brokers (eg. mosca, shiftr.io, ...) a lot. @mcollina already started with a similar API in mosca, but I guess this should be extracted and modularized.

Here is a draft:

// The manger holds the whole middleware stack
var manager = new Manager();

// Deals with pingreq packets, resets the timer on all activity
manager.use(new KeepAlive());

// Rate limiter
manager.use(new RateLimiter(
  maxBytes: 5000 // 5kb/s
));

// Treat QoS2 message as QoS1
manager.use(new DisableQoS2({
  treatAsQoS: 1
}));

// Will authorize actions
manager.use(new Authorization({
  disableAnonymous: true,
  authorizeConnect: function(client, username, password, clientId, next) {
    next(null, true);
  },
  authorizePublish: function(client, topic, payload, next) {
    next(null, true)
  },
  authorizeSubscribe: function(client, topic, next) {
    next(null, true);
  },
  authorizeForward: function(client, topic, payload, next) {
    next(null, true);
  }
}));

// Custom handler
manager.handle('publish0', function(client, topic, payload, next){
  //TODO: publish to all other clients
  next();
});

// Custom handler
manager.handle('publish1', function(client, topic, payload, next){
  //TODO: publish to all other clients
  next(null, true);
});

// Handle incoming connections
server.startServers({}, manager.handle.bind(manager), function(){
  console.log('server running');
});

A published message would go through all the middlewares in the order as they are defined.

Please share your ideas!

[RangerMauve]

Would these middleware just hook into events that are already provided by the broker?

[256dpi]

@RangerMauve No, the idea is to abstract and modularize components you need to build your own broker, which for example uses a cluster of a higher level MQ servers in the background or fully runs in RAM. The middleware stack would be similar to express that helps you build web apps and API's by providing reusable components you can chain together.

[mcollina]

I am rather against this proposal. MQTT is a standard, not really something you build-your-own-broker. Plus, you might want your broker to be FAST, and IMHO these middlewares will slow down.

The reason I think mqtt-server is helpful is as a building block (and really helpful in tests), plus it was a functionality I really want to take out of MQTT.js, which I think should serve as a client.

Anyway, let's build these couples of modules, then we might talk about middlewares.

As a side note, I am in this build-your-own-broker thing, check out my http://npm.im/mqemitter, http://npm.im/mqemitter-redis and http://npm.im/mqbroker. I just think these should not be part or mqttjs. Il giorno gio 22 gen 2015 alle 17:54 Joël Gähwiler notifications@github.com ha scritto:

This is an RFC for a middleware based management of connections. This API would simplify the development of custom brokers (eg. mosca, shiftr.io, ...) alot. @mcollina https://github.com/mcollina already started with a similar API in mosca, but I guess this should be extracted and modularized.

Here is a draft:

// The manger holds the whole middleware stackvar manager = new Manager(); // Deals with pingreq packets and closes manager.use(new KeepAlive()); // Treat QoS2 message as QoS1 manager.use(new DisableQoS2({ treatAsQoS: 1 })); // Will authorize actions manager.use(new Authorization({ disableAnonymous: true, authorizeConnect: function(client, username, password, clientId, next) { next(null, true); }, authorizePublish: function(client, topic, payload, next) { next(null, true) }, authorizeSubscribe: function(client, topic, next) { next(null, true); }, authorizeForward: function(clinet, topic, payload, next) { next(null, true); } })); // Custom handler manager.handle('publish0', function(client, topic, payload, next){ // publish to all other clients next(); }); // Custom handler manager.handle('publish1', function(client, topic, payload, next){ // publish to all other clients next(null, true); }); // Handle incoming connections server.startServers({}, manager.handle.bind(manager), function(){ console.log('server running'); });

Please share your ideas!

— Reply to this email directly or view it on GitHub https://github.com/mqttjs/mqtt-server/issues/2.

[RangerMauve]

If there is a middleware idea, I think it'd be better to have that as part of a module that would build up on this. So you could have something like:

mqtt.startServers(config, manager);

That way if people wanted the convenience of the middleware chain, they could integrate with it, otherwise it wouldn't impact the performance at all.

[mcollina]

:+1: for @RangerMauve proposal.

[256dpi]

@RangerMauve Of course such a thing should be modularized and 100% opt-in.

I started this thread because I think there should be a discussion about how building a broker can also be simplified by providing libraries and frameworks. I belief that MQTT is not only a client protocol and that building brokers is done by a few that know how to do it. IMHO it should be made as simple as possible to support new implementations and new services.

[RangerMauve]

@256dpi I totally agree, too. In one of my projects I've added a way to track the presence of users based on their credentials when they connect to the broker. It'd be nice to give people a way to reuse something like that easily.

[256dpi]

I started working on a proof of concept for this idea: https://github.com/256dpi/mqtt-stack Would be cool if you all could take a look and give some feedback!

[mcollina]

This is impressive :D. My only suggestion is to not use the data event in https://github.com/256dpi/mqtt-stack/blob/master/src/stack.js#L29-L31, but rather call pipe() passing in a Writable. In that way you can block a client that is sending too many messages.

[256dpi]

@mcollina Would you be up for creating an 'mqtt-stack' repo in mqttjs, so that we could work with PRs for such things?

[mcollina]

done :)

Il giorno Tue Jan 27 2015 at 11:24:53 Joël Gähwiler < notifications@github.com> ha scritto:

@mcollina https://github.com/mcollina Would you be up for creating an 'mqtt-stack' repo in mqttjs, so that we could work with PRs for such things?

— Reply to this email directly or view it on GitHub https://github.com/mqttjs/mqtt-server/issues/2#issuecomment-71622683.

[256dpi]

Continuing here: https://github.com/mqttjs/mqtt-stack.