Problem with UTF-8 characters in substitutional tags

[jpockels]

Subject Line is not being encoded correctly for characters outside the ASCII range. When sending emails, none of the subjects I specify with special characters (through ether the API with mail.setSubject or through the email templates) are presented correctly when using content type text/html and no substitution tags in the subject. Am I missing something that I need to specify in order for this to work as expected? Example:

The body doesn't seem to be affected by this issue. Right now I am specifying the content type of an email by using: Content("text/html", "some text here");

This seems to default to the UTF-8 charset conversion for the BODY which is fine, however it doesn't seem to affect the SUBJECT line unless I add a Substitution Tag to it or change the content type. For example: mail.setSubject("¡Feliz Cumpleaños!"); <---- Fails to do conversion. See the example above. mail.setSubject("¡%Happy_Spanish% Cumpleaños!"); <---- Displayed correctly

Content("text/plain", "some text here"); <----- Also presents the subject line correctly. (Regardless of the presence of any substitution tags). Note: I haven't tested the other content types.

Some web application such as Mailinator and Gmail would do the UTF-8 MIME content transfer encoding regardless of whether or not it was specified as a content-type. This is why only on these Web Applications the subject will display correctly.

So in summary: Using content text/html and a subject without any substitution tags, would fail to present special characters in the subject line correctly or rather fail to apply MIME content transfer encoding for UTF-8 characters.

If I understand the issue correctly, an email subject is sent as a header which requires ASCII only characters, which means it needs to be encoded using RFC 2047 specs using Quoted-Printable or Base64 such that:

=?charset?encoding?encoded-text?= e.g. =?utf-8?Q?hello?=

I could probably temporarily solve this by using Java and pre-encoding the subject before sent http://stackoverflow.com/questions/3451256/javamail-changing-charset-of-subject-line However this would not work for subjects specified in transactional templates using the Web Application. Which is the way we wanted to use it. Marketing personnel is creating and updating templates that we developers use by specifying the template id (which pulls the subject line from it as well) using the Java API.

Steps to Reproduce

1. Post JSON request with special characters using API e.g.: { "from" : { "name" : "Joaquin Pockels", "email" : "email.test2@email.com" }, "subject" : "Default Subject Line", "personalizations" : [ { "to" : [ { "email" : "email.test1@email.com" } ], "substitutions" : { "%Today_Date_Spanish__c%" : "11 DE AGOSTO DE 2016", }, "custom_args" : { "sf_contact_id" : "xxxxxxxxxxx" }, "subject": "¡Feliz Cumpleaños!" } ], "content" : [ { "type" : "text/html", "value" : "some text here" } ] }

   Technical details:

   • sendgrid-java Version: master (latest commit: [commit number])
   • Java Version: 3.0.8

[thinkingserious]

Thanks for the detailed issue report @jpockels!

We have added this to our backlog for a fix.

It can move up our queue with +1s, comments or if someone submits a pull request with a signed CLA.

[blessiedj]

Hi @thinkingserious I am working with Sendgrid.java api for google app engine. I am using transactional templates with substitutional tags. The substitution tags and subject has French characters. Mail is sent when there is no French characters like é. Could you please tell me if the issue has been resolved ?

[thinkingserious]

Hello @armis-tech,

That issue should be fixed, per: https://github.com/sendgrid/sendgrid-java/issues/139

If you are still having issues, please open a new issue. Thanks!

[blessiedj]

ok sure thanks. I will check and let you know.

[blessiedj]

Hello @thinkingserious it works with the latest jar. Thank you

[thinkingserious]

Awesome, thanks for confirming!

[thinkingserious]

@jpockels,

Are you still experiencing this issue?

[blessiedj]

@thinkingserious https://github.com/sendgrid/sendgrid-java This latest jar works well locally with eclipse java web project for transactional templates with utf-8 characters. But as i tried to deploy in google app engine environment it is not working. I am not getting any response. Also initially i was using https://github.com/sendgrid/sendgrid-google-java , the jar specially provided for google app engine with java and sendgrid. But this does not handle utf-8 characters. So i decided to use https://github.com/sendgrid/sendgrid-java. I have one more question. Does this sendgrid-google-java jar handle email with sending attachments. I could not find that in the read me. Please provide your guidance

[thinkingserious]

Hello @armis-tech,

The sendgrid-java-java library is based on the v2 mail send, and while the v2 mail send endpoint supports attachments, the library does not, so sending attachments would look something like this: https://sendgrid.com/docs/API_Reference/Web_API/mail.html#-Send-a-test-with-attachment.

Were you able to get this library (sendgrid-java) to work on Google App Engine?

[blessiedj]

Hi @thinkingserious,

Thanks for the response. As I sent mail with attachments using sendgrid-java-java library in my local environment(Web project eclipse java application) i received two emails one with the content as specified in transactional template and the other one without any content but only attachments. The library (sendgrid-java) is not working with Google App Engine. But Sendgrid-google-java works with GAE but does not support UTF-8 or attachments.

[thinkingserious]

Hello @armis-tech,

What is the error you get when trying to use sendgrid-java on GAE?

[blessiedj]

Hi @thinkingserious ,

There is no error or exceptions. I have all the request, content and mail objects created and set but after creating the response object and when i try to send email using Response response = sendgridObj.api(request), after this line I have logs for fetching response code which is not printed and my application runs smoothly till the end. I have no trace of any error.

[thinkingserious]

@armis-tech,

Thanks for the follow up. I've added this to our backlog for further investigation.

[blessiedj]

Hi @thinkingserious ,

As billing was not enabled for my Google App Engine project, my emails have not been sent. Now the application works very well in GAE with transactional templates and attachments with this jar https://github.com/sendgrid/sendgrid-java. Thanks for the response. But it has a problem on UTF-8 characters in substitutional tags. Sorry for the hassle created earlier. Thanks for the support.

[thinkingserious]

No problem @armis-tech! I'm glad you were able to get things working on GAE.

Please continue to follow this thread for a UTF-8 fix. Thanks!