search

parcelvoy / platform

Parcelvoy: Open source multi-channel marketing automation platform. Send data-driven emails, sms, push notifications and more!
https://parcelvoy.com
MIT License
258 stars 47 forks source link

Email sending speed and stats #446

Closed tanmay-predisai closed 7 months ago

tanmay-predisai commented 7 months ago

We tried sending a newsletter to 50K users via AWS SES. A couple of questions:

  1. Will Parcelvoy keep updating the sending stats on campaigns? While sending this newsletter, it was stuck on 3899/48K for a long time and finally showed the entire campaign as completed.

  2. How do we ensure that Parcelvoy is obeying the AWS SES sending rate limit? As of now, it shows the entire 48K emails sent at the same time.

    Screenshot 2024-04-12 at 6 22 28 PM

  3. This is how the CPU utilization looks for the EC2 instance. Will it stay the same (wont go beyond 80%) even if we send the newsletter to 600K recipients instead of 50K? As of now we are using a t3.large instance.

    Screenshot 2024-04-12 at 6 26 07 PM

  4. Will the server use lesser resources if we turn off the LOG_COMPILED_MESSAGE setting since we don't want each message to be logged anyways?

Thank you for your help!

pushchris commented 7 months ago
  1. Stats (the numbers up top) are only updated every five minutes so there may be a delay in what you are seeing. If you refresh the page however you should see the status of the send updating and showing the status immediately
  2. You set the rate limit for SES inside of the provider itself. The value is in messages per second. You'll see messages enter a throttled state if they aren't able to immediately send.
  3. By default jobs are processed at a concurrency of 10 simultaneously processing messages per worker container (which is typically more than enough to send thousands of messages a second). If you want to adjust this you can add the variable REDIS_CONCURRENCY to docker compose and set that value up or down. If your CPU hits a ceiling, it'll just slow down the rate of send and take longer to process sending over multiple minutes for example. If you want to increase this, you would increase your instance size and add an additional worker container to help with concurrency. Overall what you are most likely seeing is MySQL and the workers competing for CPU resources here, if your send size continues to grow we would very much recommend moving MySQL over to RDS such that you can grow the installation beyond a single instance.
  4. It may improve resource utilization, but I would not expect it to shift dramatically