cds-snc / notification-planning-core

Project planning for GC Notify Core Team

0 stars 0 forks source link

Support larger notification's message than 256k #285

Open jimleroyer opened 4 months ago

jimleroyer commented 4 months ago

Description

As a user of GCNotify, I want to send a large email, So that I can provide all necessary information to my recipients.

As a system operator of GCNotify, I want to increase the message body size support, So that I can increase the reliability of the system.

WHY are we building?

To increase the stability of the system. At current time, there is a risk that a notification customization field being too large might get stuck in the data pipeline and especially in SQS due to its limited 256kb support. Recently, SQS added support in the Python SQS library for payload support up to 2gb.

WHAT are we building?

To work natively with no modifications of our stack, we'd need celery to support natively.
Or we find some other ways to bypass celery.

VALUE created by our solution

Increased message size within GCNotify for one-var notifications or others with partials with lots of information.
More stability as we do put batched notification (with bundles of 10s) into SQS and these can take over 256kb altogether.

Acceptance Criteria

[ ] Investigation and discovery work performed on best option to support AWS extended library -- do we use Celery? do we bypass it? do we implement it? 🤔💭

QA Steps

[ ] TBD