comic / grand-challenge.org

A platform for end-to-end development of machine learning solutions in biomedical imaging
https://grand-challenge.org
Apache License 2.0
168 stars 50 forks source link

Replace django-celery-email #3310

Closed jmsmkn closed 2 months ago

jmsmkn commented 2 months ago

The project is abandonware and the retry behaviour is causing problems. Browsing the code, we would only need to:

It would still use django-ses under the hood, but that is still well maintained.

jmsmkn commented 2 months ago

The most straightforward way to do this would be to use https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ses/client/send_raw_email.html - then both dependencies could be replaced.

We could get over the celery message size limits by storing the email in the database, then sending a reference to that pk in the celery task.

jmsmkn commented 2 months ago

That was all straightforward. I'll run some tests on staging this afternoon.

jmsmkn commented 2 months ago

This appears to work well, we need to:

jmsmkn commented 2 months ago

We could now also prevent queue flooding by having a periodic task that sends the email. This could also handle the SES rate limit.