Increase SMTP recipient limit

[wsanchez]

We appear to be capped at 14 recipients, despite the documented limit being 50.

[wsanchez]

I've submitted a support request to increase the limit. Case ID is 5832831801.

[wsanchez]

Reply:

Thank you for submitting your request for a maximum sending rate increase. Your new sending quota is 100,000 messages per day and your maximum send rate is now 50 messages per second.

I can try providing more info about why we need to burst out emails more quickly.

[wsanchez]

Questions to answer if we want to try again:

-- Approximately how big will your average email be, in kilobytes? -- In what period of time do you need to be able to send your requested volume of email? For example, 1 hour, 5 hours, 24 hours. -- Why do you need to send your emails within that window (for example, the mail is an emergency broadcast, you are doing a flash sale, etc.)? -- What are the details of your use case? -- Have you implemented a mechanism for handling throttling exceptions? You can find a sample solution for handling this restriction in the following post on the AWS Messaging and Targeting blog (https://aws.amazon.com/blogs/messaging-and-targeting/how-to-handle-a-throttling-maximum-sending-rate-exceeded-error/ ).

[mikeburg]

Let me see how much overhead AWS SES has versus SMTP. If a 50 message request can complete in less than 2 seconds, we're golden. That would mean an emergency broadcast would take around 32 seconds ((800 rangers divided by 50 msgs/sec) * 2 sec overhead) to complete.

If the SES overhead is too great, the message per second limit needs to bumped higher OR we need to start building a worker queue container to handle the long processing time.

The two tests I ran during the last event with AWS SMTP and the 14 message/sec limit took over 2 1/2 minutes to complete. The browser & nginx server timed out after 30 seconds.

[flwyd]

Is increasing the nginx timeout a workable solution?

Browsers generally shouldn't time out that quickly if there's a server connection in place, and we could append status messages as the emails go out.

[A worker queue is of course The Right Way to do something like this, but it's understandably not PHP's strong suit.]

[wsanchez]

Does a client timing out cause a PHP request to stop processing?

Sending back an HTTP 100 after each batch might help…? I don't have much experience there.

[wsanchez]

I think we should look into a background job. If that's hard with Laravel, it's shouldn't be hard for the IMS to do it in the background.

The question will be how the background job is triggered. We can use polling to start with. In AWS I don't especially like that because it'll cost $something, but on playa it'll be no big deal at all.

[mikeburg]

The client nor nginx timing out should not terminate the process.. however there might be some issue with nginx trying to reconnect with a busy php process. (Not entirely sure here -- don't quote me on that. )

The background job/worker queue route is the better solution, and more flexible.

According to the Laravel document, worker queues are pretty trivial to setup and run. Create a queue table, and setup a process to run php artisan queue:work. The command loops forever polling the database for queued jobs, and executes them until the process is terminated.

Laravel's Mail framework has the concept of worker queue built into it - the calling application would say 'hey, queue this mail as a job please.'' and off it goes.

[wsanchez]

OK so in AWS, we'll need an additional service which will get updated when the API code changes. We could also just run it in each API service container, but I don't think running more than one of these is good. There's a chance they step on each other (though I'll assume they wrote it sanely and avoid that), there's no real win to running >1 unless we're throwing lots of concurrent work at the queue, and since it's polling, there's a cost to the containers spinning up occasionally even when nobody is using the service.

I'll set it up.

[wsanchez]

Moving over to #17

[wsanchez]

Closing since we solved this via #17