Error code EMFILE sending message to 19.000 devices

[cmarrero01]

HI,

I try to send a push notification to 19.000 devices each per each, and I receive the next error:

{ [Error: connect EMFILE 172.217.29.106:443 - Local (undefined:undefined)]
  code: 'EMFILE',
  errno: 'EMFILE',
  syscall: 'connect',
  address: '172.217.29.106',
  port: 443 } undefined

The code is:

for (var i = 0; i<doc.length;i++){
              sender.sendNoRetry(message, doc[i].token, function (err, result) {
                      console.log(err,result);
              });
}

None message was sent...

[eladnava]

@cmarrero01 Are you trying to send 19,000 messages with that code?

Doing it this way would que up 19,000 HTTP requests to the FCM API, which is definitely not something we want to do. The EMFILE error is thrown because the system is out of file descriptors (each socket takes up a file descriptor on Unix).

FCM allows sending to 1,000 registration IDs in the same request. You should implement a batch mechanism to split up the 19k devices and send only 19 requests via sender.sendNoRetry.

Sample implementation: https://github.com/ToothlessGear/node-gcm/issues/42#issuecomment-145379922

[cmarrero01]

Of corse that my code above is an example only, my real code is more complex but with the same result.

The thing is the message is different for each device, so, I need send each message individualy.

For send batch messages I do something like this.

`var e;
                    var j;
                    var batchs;
                    var chunk = 999;
                    for (e=0,j=notifications.length; e<j; e+=chunk) {
                        batchs = notifications.slice(e,e+chunk);
                    }

                    pushRecursivityForAndroid(0,batchs,message,sender);
function pushRecursivityForAndroid(index,batchs,message,sender){
            if(index>=batchs.length){
                console.log("Finish Batchs");
                return;
            }
            sender.sendNoRetry(message, batchs[index], function (err, result) {
                console.log(err,result);
                index++;
                pushRecursivityForAndroid(index,batchs,message,sender);
            });
        }`

[eladnava]

If each message is different, you need to implement a queue mechanism so that you don't attempt to send 19,000 requests at the same time.

async.eachLimit makes this easy (using caolan/async):

// Add all 19k messages (store recipient registration ID in message.token)
var messages = [];

// Send up to 50 messages concurrently
async.eachLimit(messages, 50,
    function (message, complete) {
        sender.sendNoRetry(message, message.token, function (err, result) {
            // Invoke async's complete() callback when done sending the notification
            complete();
        });
    },
    function (err) {
        if (err) return console.error(err);
        console.log('Done processing all messages')
    }
);

[cmarrero01]

The problem here is if take 500 ms to send 1 message, send 19.000 will take 15 minutes to send all messages, I need this on real time.

[eladnava]

@cmarrero01 Then FCM is not the right solution for your realtime requirements, especially when each message is highly individual. There's always the possibility of spinning up more servers to send more concurrent requests to FCM, but you'll need a crazy amount of servers to send 19,000 unique notifications in one second, not to mention a robust backend architecture.

There are alternative services to FCM that might be able to provide what you are looking for.

[hypesystem]

@cmarrero01 Could you consider doing the individualized lookup on the recipient devices? This would allow you to send identical messages to all devices.

The messages would then basically be "ping to fetch" messages, letting the device know that it needs to show a notification for something, but letting the device build the notification.