Node Crashes with Following Exception

[andunslg]

^[[90mPOST /publishEventPerception ^[[32m200 ^[[90m6ms^[[0m statusCode: 202 Record added as 5233d77b11b1b7087900007f Record added as 5233d77b11b1b7087900007f statusCode: 202 statusCode: 202 statusCode: 202 [2013-09-14 08:56:51,631] INFO {org.wso2.carbon.databridge.core.DataBridge} - admin connected statusCode: 202 [2013-09-14 08:56:51,705] INFO {org.wso2.carbon.databridge.core.DataBridge} - admin connected [2013-09-14 08:56:51,706] INFO {org.wso2.carbon.databridge.core.DataBridge} - admin connected [2013-09-14 08:56:51,708] INFO {org.wso2.carbon.databridge.core.DataBridge} - admin connected [2013-09-14 08:56:51,708] INFO {org.wso2.carbon.databridge.core.DataBridge} - admin connected statusCode: 202 statusCode: 202 statusCode: 202 statusCode: 202 [2013-09-14 08:56:51,727] INFO {org.wso2.carbon.databridge.core.DataBridge} - admin connected update Collection asundefined update Collection asundefined

/home/sith/Sith/Sith_Code/Sith/NodeApi/mongoAdapter.js:125 db.collection(collection,function(err,collection){ ^ TypeError: Cannot call method 'collection' of null at /home/sith/Sith/Sith_Code/Sith/NodeApi/mongoAdapter.js:125:12 at /home/sith/Sith/Sith_Code/Sith/NodeApi/node_modules/mongodb/lib/mongodb/db.js:260:16 at null. (/home/sith/Sith/Sith_Code/Sith/NodeApi/node_modules/mongodb/lib/mongodb/connection/server.js:687:7) at EventEmitter.emit (events.js:92:17) at null. (/home/sith/Sith/Sith_Code/Sith/NodeApi/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:154:15) at EventEmitter.emit (events.js:98:17) at Socket. (/home/sith/Sith/Sith_Code/Sith/NodeApi/node_modules/mongodb/lib/mongodb/connection/connection.js:492:12) at Socket.EventEmitter.emit (events.js:95:17) at TCP.close (net.js:451:12) [2013-09-14 10:56:35,351] INFO {org.apache.cassandra.cache.AutoSavingCache} - Saved KeyCache (9 items) in 44 ms [2013-09-14 10:56:57,051] INFO {org.apache.cassandra.cache.AutoSavingCache} - Saved KeyCache (10 items) in 205 ms

[andunslg]

This is the method which crashed each time,

exports.updateSelectedFields = function(collection,selector,fieldSelector){ Db(config.mongodb.database, new Server(config.mongodb.host, config.mongodb.port, {auto_reconnect: false, poolSize: 7}), {w:0, native_parser: false}).open(function(err,db){ db.collection(collection,function(err,collection){ collection.update(selector,{$set:fieldSelector},function(err,result){ if(err) throw err; console.log('update Collection as'+result); }); }); }); }

Are we closing the db connection after each operation? Or do we have open connections until node ends?

[andunslg]

Are we creating connection per each update requests?

Db(config.mongodb.database, new Server(config.mongodb.host, config.mongodb.port, {auto_reconnect: false, poolSize: 4}), {w:0, native_parser: false}).open(function(err,db){

http://stackoverflow.com/questions/12364417/node-js-mongodb-closing-the-connection http://stackoverflow.com/questions/14495975/why-its-recommended-not-to-close-any-mongodb-connection-anywhere-in-node-js-code http://stackoverflow.com/questions/16309108/mongodb-nodejs-native-driver-close-connection-or-not

[andunslg]

I locally tested the situation. And found that for 500 perceptions Node API creates 800+ connections to mongoDB. And mongo crashes saying

Thu Sep 19 22:06:03.548 [initandlisten] connection refused because too many open connections: 819

I will look in to this personally!

[andunslg]

In the mongo adapter there were places where it uses a singleton connection to DB. But there were places where it creates new connections per invocation. That caused the problem. Thus I changed them to use singleton connection.

I checked the implementation in local machine. It shows successful results. No connection failures. Thus I think we can close the issue for now. But we have to focus about load balancing in future.