Email: Google API creates garbage in an effort to "fix" the string

[Inkrementator]

Hello, the way the E-Mail Objects currently get serialized is incompatible with how google api expects mail. Some of the "fixes" they apply is interpreting multipart/alternative not as a mime-block, but just escape the header and all parts as text. The most stupid thing I've encountered was them decoding a base64 attachment and rendering it (on their server) as a string. Funnily enough, this only happens when sending E-Mails to different addresses than your own, making this an easy pitfall to catch you blindsided.

427 contains some small improvement I tried while tracking the issue down. Sadly, non of them fixed my problem.

What end up working is converting CRLF to just LF message.toString().replace("\r", "");

Which is curious, because RFC 2045 (mime messages) section 2.1 clearly states that CRLF is a requirement

The term CRLF, in this set of documents, refers to the sequence of octets corresponding to the two US-ASCII characters CR (decimal value 13) and LF (decimal value 10) which, taken together, in this order, denote a line break in RFC 822 mail.

That said, I found some old threads on the internet that encountered the same problem with different mail providers. https://stackoverflow.com/questions/6783863/what-is-the-proper-newline-in-emails-lf-or-crlf#23096310

From the php doc cited in the SO answer:

If messages are not received, try using a LF (\n) only. Some Unix mail transfer agents (most notably » qmail) replace LF by CRLF automatically (which leads to doubling CR if CRLF is used). This should be a last resort, as it does not comply with » RFC 2822.

Here is the relevant mailing list discussion that ultimately led to the inclusion of that note: https://bugs.php.net/bug.php?id=15841

The gist of the argument for omitting CR is that CRLF is only required when talking the SMTP protocol, the internal representation and when talking to other system tools like the mail program should use the system line seperator.

Some possible courses of action (roughly equivialent to what the php guys enumerated):

1. Require me do more work and reproducing this with an actual SMTP server and marking this as wontfix
2. wontfix due to message.toString().replace("\r", ""); being sufficient and not too unefficient. Maybe we should document it then
3. toString helper that accepts linesep parameter
4. toString uses system linesep by default, only using CRLF when talking to an STMP replay via EmailMessage.send

[adamdruppe]

what google api? i have always used this with an smtp server for sending.

[Inkrementator]

Google workspace [1], but on the internet, it says that the qmail program and gmx (common german email host) also has this bug. I will test with their SMTP server soon.

I don't think you can test the api without being a paying customer. I can post some garbled examples if requested but need some time to redact the headers if posting publicly.

[1] https://developers.google.com/gmail/api/reference/rest

[adamdruppe]

ok, I used to use this with a gmail smtp thing but it stopped working a while ago and i stopped keeping up with their stuff anyway.

But if we do end up having to just replace "/r", "" that's really not that bad.

[Inkrementator]

I just verified that both gmail and gmx stmp relay don't care about CRLF either way (or even a mixture of CR and CRLF) and neither does my previous pr with the 76 chars base64 line length or specifying the transfer-encoding as 8bit matter to them.

The problem with google appears only when using the API instead of smtp (which they discourage and make a bit of a pain to set up)

But if we do end up having to just replace "/r", "" that's really not that bad.

Agree. I'm more concerned with preventing somebody from having to debug this again in the feature, especially because as I said, the API behaves differently depending on whether you send to yourself or to another mail address.

[adamdruppe]

Yeah, in any case we'd have to document the crap out of it - probably on all three levels (method, class, and module) so people are more likely to see it.

[adamdruppe]

I suppose we can close now that that pr is in.