Load testing with mosca and mongodb

[uronly14me]

Hi,

Thank you for your module.

In my condition, I had 500 connected clients and 125,000 publishing requests. I used remote servers in AWS: one mosca server and one mongodb server. And when I tested, I cannot get messages after sending around 90,000 messages. And after the test, the mosca server is hanging which means it does not respond to any publishing or subscribing requests. I can connect to the server but not send or get messages.

Do you have any idea with this situation?

I follow your tips in my code,

var moscaSettings = {
  port: 1883,
  backend: {
    type: 'mongo'
    url: 'mongodb://username:password@remoteserver',
    pubsubCollection: 'pubsub',
    mongo: {}
  }
};

var server = new mosca.Server(moscaSettings);

server.on('ready', setup);

[uronly14me]

I found that without moscaSettings.backend option, the server showed better performance. I think the connection between mongodb and mosca has problem. When testing the server, database's lock percentage become high. I used mongodb for scaling.

I have tested with local redis and have same problem.

If I do not use the backend option, then does the mosca server use its memory? Can I use it without backend option?

[mcollina]

The problem is likely in https://github.com/mcollina/ascoltatori/blob/master/lib/mongo_ascoltatore.js. I think you should look into reproducing the issue just using Ascoltatori, and try to fix it there.

You should run your broker with DEBUG=ascoltatori:mongodb and send me the last 100 lines (maybe in a gist).

Also, are you using QoS 1?

If you do not specify any backend, you cannot cluster Mosca.

[uronly14me]

Thank you for your reply! And sorry for late.

I uploaded my code at gist It's my last 100 lines of log when I cannot get a message.

I have tested options with mqtt and redis. And it has same problems.

And also I use QoS 1.

[uronly14me]

Hi,

I found that mongodb capped collection was the problem. I tuned my connection settings between mongodb and mosca. It can stand a lot of load like 10k connection.

Thanks!

[mcollina]

Good to know! Can you please post your final settings, so other people that aface the same problem can reuse your solution?

[uronly14me]

I used pub-sub-settings like this.

var pubsubSettings = {
  type: 'mongo',
  uri: mongoUrl,
  db: 'couplete',
  size: 10 * 1024 * 1024 * 1024, // 10 GB
  max: 5000 * 10000, // documents
  pubsubCollection: 'pubsub',
  mongo: {}
};

[mcollina]

Thanks!!