push notifications - high throughput

[obeyer]

Could you, please, explain this limitation from the docs, like what is the number (10K, 50K, 100K, a million?): Does not support super high throughput since it does not employ a job queue system here http://docs.parseplatform.org/parse-server/guide/#push-notifications

[funkenstrahlen]

Sending push notifications is done on the same process as your nodejs web worker is running on (no background job queue!). The more notifications it does have to send, the longer it takes for the web worker to respond to new requests.

How big the impact on your performance actually is depends on the number of notifications to send and the hardware your parse server is running on.

[mortizbey]

Is there any special configuration you recommend for us to be able to send more than 10K pushes daily? Or any special adapter?

[funkenstrahlen]

Do you experience any issues with your current configuration?

You can use node clustering to run multiple workers. If one thread has high load because of push notifications you can rely on the other threads responding to requests: https://devcenter.heroku.com/articles/node-concurrency

How many threads you should optimally run depends on you hardware configuration.

[mortizbey]

We have a load balanced configuration running on AWS Elastic Beanstalk with 4 concurrent EC2 instances. I have bumped them to 8 and still the servers freeze. Definitely there may be some issue with the Push adapter that creates a lot of stress on the Parse servers.

[thisfiore]

+1 Really interested in understanding how to solve this.

I did try to build a queue on my own but I get issues with multiple notifications being sent to the same users...

I also see a big mem leak with 10k users.

[flovilmart]

Actually it now supports high throughput with a queue using

push: {
   queueOptions: {
       messageQueueAdapter: 'message-queue-adapter',
   }
}

You can then use any message queue adapter compatible with parse server which ensures a single delivery of the push work items.

[bmueller]

@flovilmart are these queue options documented anywhere? I can't find anything in the documentation for the parse-push-adapter or via google. Is a push queue enabled by default or do I have to start one manually? I've got parse-server running on Heroku, in case that makes any difference.

[flovilmart]

@bmueller this is not directly documented but you can find the initializer there: https://github.com/parse-community/parse-server/blob/master/src/Controllers/index.js#L192 and you can see the PushQueue initializer there

By default a local queue is setup so you do not have to do anything particular. In certain cases you may want to use a queue in order to spread the load for calling the push adapters on multiple servers. For achieving this, you can use a message queue adapter like the SQS.

You should consider those options if your parse server is getting way too slow when pushs are being sent.

[vincenzoarieta93]

Does anybody know how to use parse-server-sqs-mq-adaper in order to leverage SQS system and support high throughput?

[davimacedo]

Have you tried to follow the steps in the repository's README. Did you find any issue?

[vincenzoarieta93]

@davimacedo No I haven't, but I read it all. I didn't go through the implementation because I am not 100% sure that the adapter meets my needs. Do you know if parse-server-sqs-mq-adaper removes the overhead about push notifications from the server? I tried to use parse-server-sns-adapter but it didn't solve my problem, that is: Dispatch Tens of Thousands push notifications to mobile devices from Cloud Code. Can you help me with that? Thank you

[davimacedo]

The parse-server-sqs-mq-adaper can help you to spread the dispatch process workload across a cluster of machines and therefore you can have a more scalable push notification process.

[vincenzoarieta93]

@davimacedo How is that supposed to work? Can you provide any examples? Am I supposed to use lambda functions for implementing the sqs consumers? Thanks.

[acinader]

@vincenzoarieta93 I answered you here: https://github.com/parse-community/parse-server-sqs-mq-adapter/issues/18

I'm sorry that I don't have better documentation! I'd be glad to try and work through this with you.