moscajs / mosca

MQTT broker as a module
mosca.io
3.2k stars 509 forks source link

Mosca Crashing Server #525

Open ccravens opened 8 years ago

ccravens commented 8 years ago

It appears that having the Mosca library included in the server may be crashing it. At, what appears to be random intervals, we get the following:

/Users/ccravens/Business/company/Clients/project/src/project-core/node_modules/ascoltatori/node_modules/mongodb/lib/utils.js:98
    process.nextTick(function() { throw err; });
                                  ^

Error: Cannot stat if collection is capped or not
    at /Users/ccravens/Business/company/Clients/project/src/project-core/node_modules/ascoltatori/lib/mongo_ascoltatore.js:235:26
    at handleCallback (/Users/ccravens/Business/company/Clients/project/src/project-core/node_modules/ascoltatori/node_modules/mongodb/lib/utils.js:96:12)
    at /Users/ccravens/Business/company/Clients/project/src/project-core/node_modules/ascoltatori/node_modules/mongodb/lib/collection.js:1503:20
ccravens commented 8 years ago

Actually this might be due to the connection to the database being poor. Might not be an ascoltatori issue at all... Just wanted to verify everyone's thoughts before closing it.

mcollina commented 8 years ago

I think it's a combination of both. The connection is poor, and ascoltatori is failing too early: https://github.com/mcollina/ascoltatori/blob/master/lib/mongo_ascoltatore.js#L232-L237.

I think we should add a retry mechanism there.

dushyantbangal commented 7 years ago

@mcollina has there been any fix for this? can you suggest a workaround? My MQTT just stops working after this error :(

mcollina commented 7 years ago

No I have not worked on a fix. Would you like to send a PR?

dushyantbangal commented 7 years ago

I've come up with the following (dirty) retry logic:

if (err) 
{
    if(!retryCount)
        retryCount=0;
    if(retryCount<that._maxRetry)
    {
        debug('checkCappedAndPoll -> Cannot stat isCapped. Retry');

        setTimeout(function(){
            that._checkCappedAndPoll(latest, ++retryCount);
        },500)

        return;
    }else{
        debug('checkCappedAndPoll -> Cannot stat isCapped. Give up');

        that.emit('error', new Error('Cannot stat if collection is capped or not'));
        that._handlingCursorFailure = false;
        return;
    }
}

Should I be doing it like this? Should I be putting the setTimeout of 500ms or directly retrying or putting larger timeout? Should I be using something else than your that._maxRetry ?

mcollina commented 7 years ago

@dushyantbangal does that solves your problem?

I think we can increase the maximum retry count to 5 and do something like setTimeout(func, 100 * retryCount), so that it increase overtime (and the first one is 0).

dushyantbangal commented 7 years ago

I'm not really able to recreate the issue, it happens sometimes. But I did try the following:

  1. Set error=true for count 0,1,2
  2. Execute the broker with DEBUG=ascoltatori:mongodb

For counts 0,1,2 the execution went in retry, and after count 2 the polls were successful.

mcollina commented 7 years ago

Can you send a PR then?

dushyantbangal commented 7 years ago

Sure, will do!