putyourlightson / craft-campaign

Send and manage email campaigns, contacts and mailing lists in Craft CMS.
https://putyourlightson.com/plugins/campaign
Other
63 stars 25 forks source link

Huge sendouts running all in one queue job #476

Closed samuelreichor closed 6 months ago

samuelreichor commented 6 months ago

Bug Report

I don't know if there are clear arguments that all sendouts should be processed in one single queue job, but we have a problem with an SMTP provider that only 10 emails per minute can be sent out. We have turned the max batch size to 10 and there is a 60s wait between each sendout. With 5000 emails this takes a bit of time :D

If a queue job runs that long, many php limits would also have to be set higher. So we asked ourselves whether it wouldn't be smarter if each batch ran in one queue job and these were simply processed one after the other. (Similar to Transforming Images)

Plugin Version

2.15.3

Craft CMS Version

4.5.5

PHP Version

8.1

bencroker commented 6 months ago

You should really switch to a dedicated email delivery service, otherwise sendouts are going to take hours to send (as I’m sure you already know) and you won’t get bounce and complaint handling.

Each sendout is sent in its own queue job. Each batch jobs spawns a new batch job, which should look out for memory limits but not timeout issues. If you feel like the batch job system, which is part of Craft core, needs improvement, then please create a feature request at https://github.com/craftcms/cms/discussions