cds-snc / notification-planning

Project planning for GC Notify Team
5 stars 0 forks source link

Accents in service names stripped from email "Friendly From" #898

Closed amazingphilippe closed 1 year ago

amazingphilippe commented 2 years ago

Describe the bug

Accents are stripped from the "Friendly From" field in emails

Bug Severity

See examples in the documentation

SEV-3 Impacts bilingualism and recipient UX

To Reproduce

Steps to reproduce the behavior:

  1. Put accents in your service name
  2. Send yourself a test messages
  3. Accents are replaced with unaccented letters in "Friendly From"

Expected behavior

Accented letters would be better. This doesn't provide a good French UX

Impact

French service names that contain accented letters in admin won't show accents to recipients. Bad French quality.

If applicable

Impact on Notify users: Yes

Impact on Recipients: Yes

Impact on Notify team: No

Screenshots

Capture d’écran, le 2022-09-29 à 13.52.51.png Capture d’écran, le 2022-09-29 à 13.53.15.png

Additional context

Find out what are the limitations and why it was designed like this. Maybe its on purpose, maybe we can just not replace accented letters.

yaelberger-commits commented 2 years ago

Copied from duplicate card #952 Describe the bug We convert the service name to ascii when we put it in the "from field". This removes all accents.

Bug Severity See examples in the documentation

SEV-3 Minor Does not affect functionality, but it could make end users question how well the government supports bilingualism

To Reproduce Put an accent in your service name Send yourself an email See that the service name in the "from" See error Expected behavior From line has the accent in the service name

Impact End users see that services are missing accents and are annoyed.

Additional context AWS doesn't fully support unicode in email addresses, however it looks like it would be allowed in the service name. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ses.html#SES.Client.send_raw_email

Discovered via a support ticket: https://cds-snc.freshdesk.com/a/tickets/9718

QA verify that accents in the service name are in the From field notify the user who sent the support ticket that the issue has been fixed

yaelberger-commits commented 2 years ago

Jimmy recommends an ADR to explore pros and cons and constraints Some email services might not let you create a sending/receiving email address with characters or send if there are special characters

yaelberger-commits commented 2 years ago

Test creating name and email with accents and see which email services allow it Gmail, yahoo, microsoft, outlook etc.

yaelberger-commits commented 2 years ago

We won't support this, but we would need to update guidance to clients so they are aware

amazingphilippe commented 2 years ago

I understand not supporting this for the sending address. Why can't we support this for the "from" field? Which maps to the service name.

yaelberger-commits commented 2 years ago

From @jimleroyer The shared documentation in the bug description refers to this, to be explicit in what we'd have to do, I bolded the important part: Amazon SES does not support the SMTPUTF8 extension, as described in RFC6531 . For this reason, the local part of a source email address (the part of the email address that precedes the @ sign) may only contain 7-bit ASCII characters . If the domain part of an address (the part after the @ sign) contains non-ASCII characters, they must be encoded using Punycode, as described in RFC3492 . The sender name (also known as the friendly name ) may contain non-ASCII characters. These characters must be encoded using MIME encoded-word syntax, as described in RFC 2047 . MIME encoded-word syntax uses the following form: =?charset?encoding?encoded-text?= . So that wouldn't be RFC 6531/2 but RFC 2047 that we'd target, if I understand that correctly.

sastels commented 1 year ago

We might actually be able to have unicode in the email addresses... looking at https://docs.aws.amazon.com/ses/latest/dg/send-email-raw.html

For example, to send an email to 张伟@example.com, use Punycode encoding on the local part of the address (the part before the @ sign). The resulting, encoded address is xn--cpqy30b@example.com.

ramya-cds commented 1 year ago

Another Freshdesk ticket with accent in their service name - https://cds-snc.freshdesk.com/a/tickets/14757

jimleroyer commented 1 year ago

Interesting find @sastels , it seems to be using a different API (i.e. SendEmailRaw vs SendEmail that we currently use). We could test this out.

andrewleith commented 1 year ago

I think the part of the address we are talking about here is called the "Display name" so we may want to refer to it as that to reduce confusion!

smcmurtry commented 1 year ago

@andrewleith AWS refers to it as the "Friendly From" field (ctrl-f for "friendly" in this page): https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ses/client/send_raw_email.html

andrewleith commented 1 year ago

Ah ok good to know, let's stick with that then to avoid ambiguity. Thanks for sharing that!

adriannelee commented 1 year ago

Waiting till today's release to merge this.

amazingphilippe commented 1 year ago

QA'd on a gmail client, our tbs outlook on tablet, and my personal address on another email provider and client and accents are coming through just fine!

Tested the following accents and all have come through as expected: ÂâàÇÉéÈèÊêÔôÙùÛûÀçËë

smcmurtry commented 1 year ago

This is now deployed to Prod.