Custom header gets too long, gets folded at wrong places

[chocolata]

Hi

I'm using PHPMailer 5.2.13 to send out mails via SendGrid. To allow sending to multiple recipients (who don't get to see eachothers e-mailaddresses), SendGrid uses what they call X-SMTPAPI.

Basically, it's a custom SMTP header that includes a list of e-mailaddressess (for bulk e-mail purposes), some tags, some categories, a unique id, some template information... As you can imagine, this header gets quite long. The contents of this custom header is JSON.

An example of an X-SMTPAPI header is as follows: {"category":["contactform", "estimate"],"unique_args":{"token":"573dadda80dbd","from_url":"https:\/\/www.website.be"},"to":["test@test.com","another@test.com","thelistgoes@on.com"]}

Now, when the recipient list gets really long, I start to have issues. Please have a look below. You can see that the headers are broken up at a specific length, regardless if that is in the middle of an e-mail address.

(Please scroll down to see what I've tried).

2: SERVER -> CLIENT: 220 SG ESMTP service ready at ismtpd0004p1lon1.sendgrid.net
2: CLIENT -> SERVER: EHLO www.website.com
2: SERVER -> CLIENT: 250-smtp.sendgrid.net
250-8BITMIME
250-PIPELINING
250-SIZE 31457280
250-STARTTLS
250-AUTH PLAIN LOGIN
250 AUTH=PLAIN LOGIN
2: CLIENT -> SERVER: STARTTLS
2: SERVER -> CLIENT: 220 Begin TLS negotiation now
2: CLIENT -> SERVER: EHLO www.website.com
2: SERVER -> CLIENT: 250-smtp.sendgrid.net
250-8BITMIME
250-PIPELINING
250-SIZE 31457280
250-STARTTLS
250-AUTH PLAIN LOGIN
250 AUTH=PLAIN LOGIN
2: CLIENT -> SERVER: AUTH LOGIN
2: SERVER -> CLIENT: 334 VXNlcm5hbWU6
2: CLIENT -> SERVER: Y2hhbGxlbmdlbWNfcHJvZHVjdGlvbg==
2: SERVER -> CLIENT: 334 UGFzc3dvcmQ6
2: CLIENT -> SERVER: OEM2WWd4Q01xbU4y
2: SERVER -> CLIENT: 235 Authentication successful
2: CLIENT -> SERVER: MAIL FROM:<sender@test.com>
2: SERVER -> CLIENT: 250 Sender address accepted
2: CLIENT -> SERVER: RCPT TO:<sender@test.com>
2: SERVER -> CLIENT: 250 Recipient address accepted
2: CLIENT -> SERVER: DATA
2: SERVER -> CLIENT: 354 Continue
2: CLIENT -> SERVER: Date: Thu, 19 May 2016 11:07:04 +0200
2: CLIENT -> SERVER: To: Website <sender@test.com>
2: CLIENT -> SERVER: From: Test Person <sender@test.com>
2: CLIENT -> SERVER: Reply-To: Test Person <sender@test.com>
2: CLIENT -> SERVER: Subject: Testsubject
2: CLIENT -> SERVER: Message-ID: <sender@test.com>
2: CLIENT -> SERVER: X-Mailer: PHPMailer 5.2.13 (https://github.com/PHPMailer/PHPMailer)
2: CLIENT -> SERVER: x-smtpapi:
2: CLIENT -> SERVER:    {"category":["https:\/\/www.website.com","wysiwyg"],"unique_args":{"token":"573d8238052b7","from_url":"https:\/\/www.website.com"},"to":["recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","r
2: CLIENT -> SERVER:    ecipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test
2: CLIENT -> SERVER:    .be","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","
2: CLIENT -> SERVER:    recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","
2: CLIENT -> SERVER:    recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","ar
2: CLIENT -> SERVER:    recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com","recipient@test.com"]}
2: CLIENT -> SERVER: MIME-Version: 1.0
2: CLIENT -> SERVER: Content-Type: multipart/alternative;
2: CLIENT -> SERVER:    boundary="b1_7684bf9d5df6a61314240d8e95c4f11d"
2: CLIENT -> SERVER: Content-Transfer-Encoding: 8bit
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: This is a multi-part message in MIME format.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: --b1_7684bf9d5df6a61314240d8e95c4f11d
2: CLIENT -> SERVER: Content-Type: text/plain; charset=iso-8859-1
2: CLIENT -> SERVER: Content-Transfer-Encoding: 8bit
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: Test subject 20-05-2016Hallo iedereen
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: 

As you can see, there are breaks inside email addresses causing issues...

So, I tried to manually insert a line break at a specific length (72, as per the RFC recommendation), but in that case my JSON header is no longer accepted by SendGrid.

The first thing I tried was to put spaces behind the comma's, so that the PHP word-wrap function can do its thing:

$sendgrid_options = wordwrap(str_replace(',', ', ',json_encode($sendgrid_options)), 72, "\n", false);
$mail->addCustomHeader('x-smtpapi', $sendgrid_options); // Metadata + sender info

However, this makes SendGrid drop the SMTPAPI header alltogether, even though JSON validation tools say that the syntax is correct.

Please have a look here: [https://sendgrid.com/docs/API_Reference/SMTP_API/using_the_smtp_api.html]

SendGrid does mention something about wrapping the header, but the code example is in PERL, so I'm lost:

Headers must be wrapped to keep the line length under 72. By RFC 821 no line can be longer than 1,000, so if you are going to generate this string yourself it is a good idea to make sure that you wrap it.

use strict;
use JSON;

my $header = { to => ['ben@sendgrid.com', 'joe@sendgrid.com'],
sub => { '%name%' => [ 'Ben', 'Joe' ], '%role%' =>[ 'sellerSection', 'buyerSection' ] },
section => { '%sellerSection%' => 'Seller information for: %name%', '%buyerSection%' => 'Buyer information for: %name%' },
category => 'Orders',
unique_args => { 'orderNumber' => '12345', 'eventID' => '6789' },
filters => { 'footer' => {'settings' => {'text/plain' => "Thank you for your business"}}}};

my $json = JSON->new;
$json->space_before(1);
$json->space_after(1);
my $js = $json->encode($header);
# This regex breaks the string up at whitespaces to keep the line length short
$js =~ s/(.{1,72})(\s)/$1\n   /g;
my $hdr = "X-SMTPAPI: $js";
print "$hdr\n";

So at this moment I was investigating the PHPMailer encodeHeader function, but I'm not sure that this would help.

Please advise and thank you in advance.

Here is the log once again for readability. smtp-log-new.txt

[Synchro]

That header looks absolutely correct! It's using header folding as per RFC5322 section 2.2.1. For example it could split a line like this:

header: longlinevalue<CRLF> -> header: long<CRLF><whitespace>line<CRLF><whitespace>value<CRLF>

Which looks like:

header: longlinevalue
->
header: long
 line
 value

At the receiving end, sendgrid should be applying the reverse transformation (the sequence <CRLF><whitespace><text> is replace by <text> within a message header section) before trying to use the value. This is a lossless transformation and can cope with any length line.

In your example, note that the Content-type header is doing the same thing, and is also correct.

To be clear, the problem described in #36 is that PHPMailer only looks at the header's value, and does not include the label, so if you get a value close to the hard 998-char limit, the additional length of the header's label might result in a line that's too long. This is generally fairly unlikely to happen, so it's not a priority to fix.

So, overall, I don't think that this is the cause of the problem you're having.

[chocolata]

Thanks for your in-depth response. Agreed that SendGrid should reconvert the folded header in an orderly fashion, but at this time they do not, so they suggest wrapping the header manually...

So the only thing I can go on is their PERL snippet:

# This regex breaks the string up at whitespaces to keep the line length short
$js =~ s/(.{1,72})(\s)/$1\n   /g;

However, I do not understand PERL nor have the ability to convert it into a valid PHP regex. Additionally, in my JSON-header string (especially the part with the recipient list), there are no spaces...

Do you have any suggestions on how to succesfully workaround this issue? I understand this is beyond the scope of PHPMailer.

[Synchro]

PHPMailer is already doing the same thing as that regex - they are assuming you're using a dumb script that doesn't know about mail headers. Reading the regex (which would be the same in PHP because PHP uses PCRE), it says, look for 1 - 72 characters followed by a space (greedy by default so it will match as much as it can), and replace it with the text you found, followed by a line break and two spaces. PHPMailer has a more sophisticated approach that copes with content without spaces; the sendgrid regex won't work if your content has no spaces, in fact it will probably crash as it will cause infinite backtracking.

I wrote a little script that does header unwrapping according to the RFC and it works fine:

$msg = 'complete received RFC822 email message in here';
//Detect line break format
$le = "\n";
if (strpos("\r\n", $msg) !== false) {
    $le = "\r\n";
}
//Split message into header and body
$parts = explode($le.$le, $msg, 2);
if (count($parts) != 2) {
    echo $msg; // Do nothing
}
//Unwrap headers
$parts[0] = preg_replace('/'.preg_quote($le).'\s(.*)/', '$1', $parts[0]);
//Reassemble message
echo $parts[0] . $le . $le. $parts[1];

When I send a message (using PHPMailer) with your large header in, then decode it using that script, I end up with what I started with, with no unexpected line breaks.

Can you get any feedback from sendgrid about what they don't like?

I've also discovered that PHPMailer actually already has a workaround for #36 (which I probably wrote!) - it always wraps immediately after the header label, thus ensuring that lines in the value portion can never exceed the max length.

[chocolata]

Hi, thanks for following up. I've did some tests, and they do seem to work. As per your suggestion, I've used the encodeHeader function. I didn't find any clear examples online, so I tried to do some guesswork. Here's the code I used. Could you have a look?

$sendgrid_options = str_replace(',', ', ',json_encode($sendgrid_options));
$mail->addCustomHeader('x-smtpapi', $mail->encodeHeader($sendgrid_options,72)); 

Is this the correct use of the encodeHeader function? Do you have any further suggestions?

When using this piece of code, the mail is being sent without issue. Also, upon investigating the SMTP log, I see that the folding doesn't happen inside e-mailaddresses anymore. See below:

2: SERVER -> CLIENT: 220 SG ESMTP service ready at ismtpd0004p1lon1.sendgrid.net
2: CLIENT -> SERVER: EHLO cmcrm.chocolata.be
2: SERVER -> CLIENT: 250-smtp.sendgrid.net
250-8BITMIME
250-PIPELINING
250-SIZE 31457280
250-STARTTLS
250-AUTH PLAIN LOGIN
250 AUTH=PLAIN LOGIN
2: CLIENT -> SERVER: STARTTLS
2: SERVER -> CLIENT: 220 Begin TLS negotiation now
2: CLIENT -> SERVER: EHLO cmcrm.chocolata.be
2: SERVER -> CLIENT: 250-smtp.sendgrid.net
250-8BITMIME
250-PIPELINING
250-SIZE 31457280
250-STARTTLS
250-AUTH PLAIN LOGIN
250 AUTH=PLAIN LOGIN
2: CLIENT -> SERVER: AUTH LOGIN
2: SERVER -> CLIENT: 334 VXNlcm5hbWU6
2: CLIENT -> SERVER: xxx=
2: SERVER -> CLIENT: 334 UGFzc3dvcmQ6
2: CLIENT -> SERVER: xxx
2: SERVER -> CLIENT: 235 Authentication successful
2: CLIENT -> SERVER: MAIL FROM:<noreply@website.be>
2: SERVER -> CLIENT: 250 Sender address accepted
2: CLIENT -> SERVER: RCPT TO:<to@address.com>
2: SERVER -> CLIENT: 250 Recipient address accepted
2: CLIENT -> SERVER: DATA
2: SERVER -> CLIENT: 354 Continue
2: CLIENT -> SERVER: Date: Fri, 20 May 2016 10:34:39 +0200
2: CLIENT -> SERVER: To: Website <to@address.com>
2: CLIENT -> SERVER: From: Website <noreply@website.be>
2: CLIENT -> SERVER: Reply-To: To Name <info@website.com>
2: CLIENT -> SERVER: Subject: Systeemtest, header wrapping
2: CLIENT -> SERVER: Message-ID: <c06f1bda48f949bc887e186fabb584a9@cmcrm.chocolata.be>
2: CLIENT -> SERVER: X-Mailer: PHPMailer 5.2.13 (https://github.com/PHPMailer/PHPMailer)
2: CLIENT -> SERVER: x-smtpapi: {"category":["https:\/\/cmcrm.chocolata.be", "wysiwyg"], "unique_args":{"token":"573ecc1f28bc7", "from_url":"https:\/\/www.website.be"}, "to":["test@chocolata.be", "test01@chocolata.be", "test1@chocolata.be", "test2@chocolata.be", "test3@chocolata.be", "test4@chocolata.be", "test5@chocolata.be", "test6@chocolata.be", "test7@chocolata.be", "test8@chocolata.be", "test9@chocolata.be", "test10@chocolata.be", "test11@chocolata.be", "test12@chocolata.be", "test13@chocolata.be", "test14@chocolata.be", "test15@chocolata.be", "test16@chocolata.be", "test17@chocolata.be", "test18@chocolata.be", "test19@chocolata.be", "test20@chocolata.be", "test21@chocolata.be", "test22@chocolata.be", "test23@chocolata.be", "test24@chocolata.be", "test25@chocolata.be", "test26@chocolata.be",
2: CLIENT -> SERVER:    "test27@chocolata.be", "test28@chocolata.be", "test29@chocolata.be", "test30@chocolata.be", "test31@chocolata.be", "test32@chocolata.be", "test33@chocolata.be", "test34@chocolata.be", "test35@chocolata.be", "test36@chocolata.be", "test37@chocolata.be", "test38@chocolata.be", "test39@chocolata.be", "test40@chocolata.be", "test41@chocolata.be", "test42@chocolata.be", "test43@chocolata.be", "test44@chocolata.be", "test45@chocolata.be", "test46@chocolata.be", "test47@chocolata.be", "test48@chocolata.be", "test49@chocolata.be", "test50@chocolata.be", "test51@chocolata.be", "test52@chocolata.be", "test53@chocolata.be", "test54@chocolata.be", "test55@chocolata.be", "test56@chocolata.be", "test57@chocolata.be", "test58@chocolata.be", "test59@chocolata.be", "test60@chocolata.be", "test61@chocolata.be", "test62@chocolata.be", "test63@chocolata.be", "test64@chocolata.be", "test65@chocolata.be", "test66@chocolata.be", "test67@chocolata.be", "test68@chocolata.be", "test69@chocolata.be",
2: CLIENT -> SERVER:    "test70@chocolata.be", "test71@chocolata.be", "test72@chocolata.be", "test73@chocolata.be", "test74@chocolata.be", "test75@chocolata.be", "test76@chocolata.be", "test77@chocolata.be", "test78@chocolata.be", "test79@chocolata.be", "test80@chocolata.be", "test81@chocolata.be", "test82@chocolata.be", "test83@chocolata.be", "test84@chocolata.be", "test85@chocolata.be", "test86@chocolata.be", "test87@chocolata.be", "test88@chocolata.be", "test89@chocolata.be", "test90@chocolata.be", "test91@chocolata.be", "test92@chocolata.be", "test93@chocolata.be", "test94@chocolata.be", "test95@chocolata.be", "test96@chocolata.be", "test97@chocolata.be", "test98@chocolata.be", "test99@chocolata.be", "test100@chocolata.be", "test101@chocolata.be", "test102@chocolata.be", "test103@chocolata.be", "test104@chocolata.be", "test105@chocolata.be", "test106@chocolata.be", "test107@chocolata.be", "test108@chocolata.be", "test109@chocolata.be", "test110@chocolata.be", "test111@chocolata.be",
2: CLIENT -> SERVER:    "test112@chocolata.be", "test113@chocolata.be", "test114@chocolata.be", "test115@chocolata.be", "test116@chocolata.be", "test117@chocolata.be", "test118@chocolata.be", "test119@chocolata.be", "test120@chocolata.be", "test121@chocolata.be", "test122@chocolata.be", "test123@chocolata.be", "test124@chocolata.be", "test125@chocolata.be", "test126@chocolata.be", "test127@chocolata.be", "test128@chocolata.be", "test129@chocolata.be", "test130@chocolata.be", "test131@chocolata.be", "test132@chocolata.be", "test133@chocolata.be", "test134@chocolata.be", "test135@chocolata.be", "test136@chocolata.be", "test137@chocolata.be", "test138@chocolata.be", "test139@chocolata.be", "test140@chocolata.be", "test141@chocolata.be", "test142@chocolata.be", "test143@chocolata.be", "test144@chocolata.be", "test145@chocolata.be", "test146@chocolata.be", "test147@chocolata.be", "test148@chocolata.be", "test149@chocolata.be", "test150@chocolata.be", "test151@chocolata.be", "test152@chocolata.be",
2: CLIENT -> SERVER:    "test153@chocolata.be", "test154@chocolata.be", "test155@chocolata.be", "test156@chocolata.be", "test157@chocolata.be", "test158@chocolata.be", "test159@chocolata.be", "test160@chocolata.be", "test161@chocolata.be", "test162@chocolata.be", "test163@chocolata.be", "test164@chocolata.be", "test165@chocolata.be", "test166@chocolata.be", "test167@chocolata.be", "test168@chocolata.be", "test169@chocolata.be", "test170@chocolata.be", "test171@chocolata.be", "test172@chocolata.be", "test173@chocolata.be", "test174@chocolata.be", "test175@chocolata.be"]}
2: CLIENT -> SERVER: MIME-Version: 1.0
2: CLIENT -> SERVER: Content-Type: multipart/alternative;
2: CLIENT -> SERVER:    boundary="b1_c06f1bda48f949bc887e186fabb584a9"
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: This is a multi-part message in MIME format.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: --b1_c06f1bda48f949bc887e186fabb584a9
2: CLIENT -> SERVER: Content-Type: text/plain; charset=iso-8859-1
2: CLIENT -> SERVER: Content-Transfer-Encoding: quoted-printable
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: Systeemtest, header wrappingLorem ipsum dolor sit amet, consectetur=0Aadipi=
2: CLIENT -> SERVER: scing elit. Curabitur dui erat, viverra sit amet tellus et,=0Aposuere matti=
2: CLIENT -> SERVER: s libero. Vivamus lectus massa, interdum sit amet libero=0Aat, ullamcorper =
2: CLIENT -> SERVER: vehicula sem. In sit amet est facilisis, euismod felis=0Anon, molestie maur=
2: CLIENT -> SERVER: is. Aliquam varius ac velit ac vulputate. Fusce=0Atellus turpis, laoreet id=
2: CLIENT -> SERVER:  erat egestas, malesuada pellentesque metus.=0ADonec consequat velit in sus=
2: CLIENT -> SERVER: cipit ullamcorper. Donec ut imperdiet=0Aodio. Cras quis dictum orci, et lao=
2: CLIENT -> SERVER: reet felis. Pellentesque gravida ex=0Amassa, ornare hendrerit mi vehicula e=
2: CLIENT -> SERVER: t. Suspendisse vehicula ex quis=0Aleo congue accumsan faucibus vel erat. In=
2: CLIENT -> SERVER:  bibendum dui accumsan arcu=0Apellentesque, quis bibendum mi maximus. Morbi=
2: CLIENT -> SERVER:  iaculis fringilla=0Aaliquam. Cras in arcu tristique, sodales turpis non, s=
2: CLIENT -> SERVER: uscipit dolor.=0ADonec dapibus, turpis eget dictum lobortis, mi ex finibus =
2: CLIENT -> SERVER: tellus, at=0Afringilla purus mauris nec velit.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: Ut lacinia dui eu ex euismod venenatis. Phasellus feugiat arcu quis=0Amauri=
2: CLIENT -> SERVER: s cursus hendrerit. Proin a ante maximus, cursus lacus non,=0Aeleifend lect=
2: CLIENT -> SERVER: us. Nulla ac congue sapien. Vivamus at rhoncus lectus,=0Asit amet eleifend =
2: CLIENT -> SERVER: est. Suspendisse turpis nisi, consectetur ut=0Avestibulum ut, varius at urn=
2: CLIENT -> SERVER: a. Maecenas placerat dolor hendrerit,=0Avolutpat metus a, dignissim est.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: Vestibulum tempus interdum risus non ultrices. Maecenas egestas lacus=0Aid =
2: CLIENT -> SERVER: mauris placerat viverra. Praesent tempor cursus lectus, non=0Acondimentum n=
2: CLIENT -> SERVER: ulla vehicula accumsan. Morbi ac elementum nisi, eu=0Aultrices felis. Cras =
2: CLIENT -> SERVER: rhoncus nibh urna, quis dignissim lectus euismod=0Avitae. Curabitur faucibu=
2: CLIENT -> SERVER: s faucibus ex ullamcorper rutrum. Mauris=0Agravida elit at eros maximus, in=
2: CLIENT -> SERVER:  luctus odio gravida.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: Proin suscipit dui turpis, in semper arcu aliquet eget. Etiam pharetra=0Atr=
2: CLIENT -> SERVER: istique rhoncus. Ut magna ipsum, molestie et justo vitae, laoreet=0Amolesti=
2: CLIENT -> SERVER: e arcu. Phasellus at lacus quis nisl lacinia lobortis. Aliquam=0Aeleifend t=
2: CLIENT -> SERVER: empor lorem, eget condimentum mauris accumsan sit amet.=0AMorbi cursus orci=
2: CLIENT -> SERVER:  eget turpis ornare, eget blandit purus auctor. Nulla=0Anec nibh in velit p=
2: CLIENT -> SERVER: orttitor faucibus eget id est. Morbi ac purus=0Acommodo felis faucibus semp=
2: CLIENT -> SERVER: er. Vestibulum rutrum lacus condimentum=0Aligula iaculis, at tristique tell=
2: CLIENT -> SERVER: us dictum. Curabitur pharetra erat=0Asit amet felis ultrices volutpat. Pell=
2: CLIENT -> SERVER: entesque feugiat ut diam sit=0Aamet aliquet. Curabitur laoreet egestas arcu=
2: CLIENT -> SERVER: , vel feugiat lectus=0Atempus ac. In sollicitudin sit amet nulla in feugiat=
2: CLIENT -> SERVER: ..
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: In hac habitasse platea dictumst. Maecenas vitae porta lectus, vel=0Asceler=
2: CLIENT -> SERVER: isque orci. Suspendisse semper vehicula velit nec vestibulum.=0AAliquam tem=
2: CLIENT -> SERVER: por lacus eget tincidunt porta. Pellentesque non laoreet=0Anulla. Curabitur=
2: CLIENT -> SERVER:  eu ante lobortis, tincidunt arcu vel, sodales neque.=0AInteger ut diam nec=
2: CLIENT -> SERVER:  diam consequat vehicula. Sed ac tortor id dui=0Arutrum tristique. Nunc ege=
2: CLIENT -> SERVER: t ligula sed velit pharetra dapibus. Integer=0Aeu metus sed urna fringilla =
2: CLIENT -> SERVER: fermentum. Quisque id rutrum nulla. Ut sem=0Adui, dapibus id ex in, sagitti=
2: CLIENT -> SERVER: s lobortis nibh.
2: CLIENT -> SERVER: Met vriendelijke groeten - Cordialement - Kind regards Website=0AMCWEB: h=
2: CLIENT -> SERVER: ttp://planet.website.bePHONE: 03 650 09 80MAIL:=0Ahello@website.b=
2: CLIENT -> SERVER: eADDRESS: Ter Borcht 46 - 2930 Brasschaat
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: --b1_c06f1bda48f949bc887e186fabb584a9
2: CLIENT -> SERVER: Content-Type: text/html; charset=us-ascii
2: CLIENT -> SERVER: Content-Transfer-Encoding: quoted-printable
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <h1 style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 24px; li=
2: CLIENT -> SERVER: ne-height: 24px; margin-bottom: 20px;">Systeemtest, header wrapping</h1><p>=
2: CLIENT -> SERVER: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Curabitur dui erat=
2: CLIENT -> SERVER: , viverra sit amet tellus et, posuere mattis libero. Vivamus lectus massa, =
2: CLIENT -> SERVER: interdum sit amet libero at, ullamcorper vehicula sem. In sit amet est faci=
2: CLIENT -> SERVER: lisis, euismod felis non, molestie mauris. Aliquam varius ac velit ac vulpu=
2: CLIENT -> SERVER: tate. Fusce tellus turpis, laoreet id erat egestas, malesuada pellentesque =
2: CLIENT -> SERVER: metus. Donec consequat velit in suscipit ullamcorper. Donec ut imperdiet od=
2: CLIENT -> SERVER: io. Cras quis dictum orci, et laoreet felis. Pellentesque gravida ex massa,=
2: CLIENT -> SERVER:  ornare hendrerit mi vehicula et. Suspendisse vehicula ex quis leo congue a=
2: CLIENT -> SERVER: ccumsan faucibus vel erat. In bibendum dui accumsan arcu pellentesque, quis=
2: CLIENT -> SERVER:  bibendum mi maximus. Morbi iaculis fringilla aliquam. Cras in arcu tristiq=
2: CLIENT -> SERVER: ue, sodales turpis non, suscipit dolor. Donec dapibus, turpis eget dictum l=
2: CLIENT -> SERVER: obortis, mi ex finibus tellus, at fringilla purus mauris nec velit.</p>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <p>Ut lacinia dui eu ex euismod venenatis. Phasellus feugiat arcu quis maur=
2: CLIENT -> SERVER: is cursus hendrerit. Proin a ante maximus, cursus lacus non, eleifend lectu=
2: CLIENT -> SERVER: s. Nulla ac congue sapien. Vivamus at rhoncus lectus, sit amet eleifend est=
2: CLIENT -> SERVER: .. Suspendisse turpis nisi, consectetur ut vestibulum ut, varius at urna. Ma=
2: CLIENT -> SERVER: ecenas placerat dolor hendrerit, volutpat metus a, dignissim est.</p>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <p>Vestibulum tempus interdum risus non ultrices. Maecenas egestas lacus id=
2: CLIENT -> SERVER:  mauris placerat viverra. Praesent tempor cursus lectus, non condimentum nu=
2: CLIENT -> SERVER: lla vehicula accumsan. Morbi ac elementum nisi, eu ultrices felis. Cras rho=
2: CLIENT -> SERVER: ncus nibh urna, quis dignissim lectus euismod vitae. Curabitur faucibus fau=
2: CLIENT -> SERVER: cibus ex ullamcorper rutrum. Mauris gravida elit at eros maximus, in luctus=
2: CLIENT -> SERVER:  odio gravida.</p>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <p>Proin suscipit dui turpis, in semper arcu aliquet eget. Etiam pharetra t=
2: CLIENT -> SERVER: ristique rhoncus. Ut magna ipsum, molestie et justo vitae, laoreet molestie=
2: CLIENT -> SERVER:  arcu. Phasellus at lacus quis nisl lacinia lobortis. Aliquam eleifend temp=
2: CLIENT -> SERVER: or lorem, eget condimentum mauris accumsan sit amet. Morbi cursus orci eget=
2: CLIENT -> SERVER:  turpis ornare, eget blandit purus auctor. Nulla nec nibh in velit porttito=
2: CLIENT -> SERVER: r faucibus eget id est. Morbi ac purus commodo felis faucibus semper. Vesti=
2: CLIENT -> SERVER: bulum rutrum lacus condimentum ligula iaculis, at tristique tellus dictum. =
2: CLIENT -> SERVER: Curabitur pharetra erat sit amet felis ultrices volutpat. Pellentesque feug=
2: CLIENT -> SERVER: iat ut diam sit amet aliquet. Curabitur laoreet egestas arcu, vel feugiat l=
2: CLIENT -> SERVER: ectus tempus ac. In sollicitudin sit amet nulla in feugiat.</p>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <p>In hac habitasse platea dictumst. Maecenas vitae porta lectus, vel scele=
2: CLIENT -> SERVER: risque orci. Suspendisse semper vehicula velit nec vestibulum. Aliquam temp=
2: CLIENT -> SERVER: or lacus eget tincidunt porta. Pellentesque non laoreet nulla. Curabitur eu=
2: CLIENT -> SERVER:  ante lobortis, tincidunt arcu vel, sodales neque. Integer ut diam nec diam=
2: CLIENT -> SERVER:  consequat vehicula. Sed ac tortor id dui rutrum tristique. Nunc eget ligul=
2: CLIENT -> SERVER: a sed velit pharetra dapibus. Integer eu metus sed urna fringilla fermentum=
2: CLIENT -> SERVER: .. Quisque id rutrum nulla. Ut sem dui, dapibus id ex in, sagittis lobortis =
2: CLIENT -> SERVER: nibh.</p>
2: CLIENT -> SERVER: <p style=3D"font-family: Arial, Helvetica, sans-serif; font-size: 12px; lin=
2: CLIENT -> SERVER: e-height: 20px; margin-bottom: 20px;">Met vriendelijke groeten - Cordialeme=
2: CLIENT -> SERVER: nt - Kind regards <br />Website MC</p><table style=3D"border: none; font-=
2: CLIENT -> SERVER: family: Arial, Helvetica, sans-serif; font-size: 12px; line-height: 20px; m=
2: CLIENT -> SERVER: argin-bottom: 20px;"><tr><td style=3D"padding-right: 20px;"><img src=3D"htt=
2: CLIENT -> SERVER: p://planet.website.be/img/logo.png" alt=3D"Website MC logo" /></td><=
2: CLIENT -> SERVER: td>WEB: <a style=3D"color: #CA0A42;" href=3D"http://planet.website.be"=
2: CLIENT -> SERVER:  title=3D"Visit Website MC Planet">http://planet.website.be</a><br /=
2: CLIENT -> SERVER: >PHONE: <a style=3D"color: #CA0A42;" href=3D"tel:036500980" class=3D"cicon =
2: CLIENT -> SERVER: telephone">03 650 09 80</a><br />MAIL: <a style=3D"color: #CA0A42;" href=3D=
2: CLIENT -> SERVER: "mailto:hello@website.be" class=3D"cicon email">hello@website.be<=
2: CLIENT -> SERVER: /a><br />ADDRESS: <span class=3D"cicon address">Ter Borcht 46 - 2930 Brassc=
2: CLIENT -> SERVER: haat</span></td></tr></table>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: --b1_c06f1bda48f949bc887e186fabb584a9--
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: .
2: SERVER -> CLIENT: 250 Ok: queued as zUQds7mMR2yE7EbZjTiRaQ
2: CLIENT -> SERVER: QUIT
2: SERVER -> CLIENT: 221 See you later

See the enclosed file for your convenience. new_log.txt

[Synchro]

You shouldn't need to call encodeHeader yourself at all, just set the value you want and PHPMailer will encode it for you. The second param to encodeHeader is not a length, it's a position/context name that affects how the encoding is done, so putting 72 in there will not do anything useful.

PHPMailer doesn't try to split headers on white space because it doesn't need to - header folding/unfolding is lossless so it doesn't matter where it is split. It really sounds like sendgrid is decoding the header incorrectly, which I find quite surprising. Saying that there's a problem with addresss being split in folded headers is a bit like saying that zipping a text file makes it hard to read - it doesn't matter, it's what it looks like after decoding that counts.

[chocolata]

You are completely right, I understand. However, in the short term, I need to find some kind of solution. So, what I am seeing (the correct, manual folding of the XSMTPAPI header) has nothing to do with encodeHeader? How come the header is folded at not arbitrary places after calling encodeHeader then?

You mentioned something:

You shouldn't need to call encodeHeader yourself at all, just set the value you want and PHPMailer will encode it for you.

You mean without any extra settings, it should just work, right..? Frustrating...

[Synchro]

encodeHeader only does folding for headers that need B or Q header encodings, not plain ones. Line length was not a limitation in RFC822 (message format), it came from RFC821, SMTP, (message transport). That definition was changed in later RFCs, but it's academic - the way header folding works remains the same across all the encodings and no matter when it's done.

I checked and I was wrong about it not looking for spaces - it does, but down at the SMTP level, after message generation, and just uses hard breaks if there are none, as breaking on spaces is purely cosmetic. You can see the code in SMTP::data(). If the folding has been done beforehand, it doesn't do anything.

Something you could try is to force it to use Q-encoding by including an 8-bit character anywhere in your submission - if you tack on a json property like "a":"é", that will force it to use Q encoding, which looks quite different and folds differently, and you may find that sendgrid likes it!

[chocolata]

Hi, thanks for your additional comments. I've now included an 8-bit character.

"unique_args":{"token":"5742f12e669a2", "from_url":"https:\/\/sender.domain.com", "a":"é"}

This is the setup now:

// Some custom encoding, to avoid sendgrid issues!
$sendgrid_options = str_replace(',', ', ',json_encode($sendgrid_options));
$mail->addCustomHeader('x-smtpapi', $mail->encodeHeader($sendgrid_options)); // Metadata + sender info

Which outputs to (please note the XSMTPAPI header with the extra character.

2: SERVER -> CLIENT: 220 SG ESMTP service ready at ismtpd0001p1lon1.sendgrid.net
2: CLIENT -> SERVER: EHLO sender.domain.com
2: SERVER -> CLIENT: 250-smtp.sendgrid.net
250-8BITMIME
250-PIPELINING
250-SIZE 31457280
250-STARTTLS
250-AUTH PLAIN LOGIN
250 AUTH=PLAIN LOGIN
2: CLIENT -> SERVER: STARTTLS
2: SERVER -> CLIENT: 220 Begin TLS negotiation now
2: CLIENT -> SERVER: EHLO sender.domain.com
2: SERVER -> CLIENT: 250-smtp.sendgrid.net
250-8BITMIME
250-PIPELINING
250-SIZE 31457280
250-STARTTLS
250-AUTH PLAIN LOGIN
250 AUTH=PLAIN LOGIN
2: CLIENT -> SERVER: AUTH LOGIN
2: SERVER -> CLIENT: 334 VXNlcm5hbWU6
2: CLIENT -> SERVER: xxx=
2: SERVER -> CLIENT: 334 UGFzc3dvcmQ6
2: CLIENT -> SERVER: xxx
2: SERVER -> CLIENT: 235 Authentication successful
2: CLIENT -> SERVER: MAIL FROM:<noreply@website.be>
2: SERVER -> CLIENT: 250 Sender address accepted
2: CLIENT -> SERVER: RCPT TO:<recipient@gmail.com>
2: SERVER -> CLIENT: 250 Recipient address accepted
2: CLIENT -> SERVER: DATA
2: SERVER -> CLIENT: 354 Continue
2: CLIENT -> SERVER: Date: Mon, 23 May 2016 14:01:50 +0200
2: CLIENT -> SERVER: To:  Rocket <recipient@gmail.com>
2: CLIENT -> SERVER: From: Sender Sender <noreply@website.be>
2: CLIENT -> SERVER: Reply-To: Sender Sender <sender@gmail.com>
2: CLIENT -> SERVER: Subject: Feunz
2: CLIENT -> SERVER: Message-ID: <07e25a3daab58851cf911860d31022b0@sender.domain.com>
2: CLIENT -> SERVER: X-Mailer: PHPMailer 5.2.13 (https://github.com/PHPMailer/PHPMailer)
2: CLIENT -> SERVER: x-smtpapi: {"category":["https:\/\/sender.domain.com", "wysiwyg"], "unique_args":{"token":"5742f12e669a2", "from_url":"https:\/\/sender.domain.com", "a":"\u00e9"}, "to":["lorem@recipient.com", "ipsum@recipient.com", "dolor@recipient.com", "sit@recipient.com", "amet12345@recipient.com", "consectetur@recipient.com", "adipiscing@recipient.com", "elit.test@recipient.com", "nulla@recipient.com", "placerat12345@recipient.com", "arcu@recipient.com", "quis@recipient.com", "tempus@recipient.com", "vestibulum12345@recipient.com", "felis@recipient.com", "tellus@recipient.com", "tempus@recipient.com", "arcu12345@recipient.com", "vel@recipient.com", "mollis@recipient.com", "ex@recipient.com", "est@recipient.com", "a@recipient.com", "elit.test@recipient.com", "sed@recipient.com", "a@recipient.com", "finibus@recipient.com", "nisi12345@recipient.com", "ultrices@recipient.com", "mollis@recipient.com", "ante.test@recipient.com", "suspendisse@recipient.com", "porta@recipient.com", "ante@recipient.com", "vitae@recipient.com",
2: CLIENT -> SERVER:    "quam@recipient.com", "posuere12345@recipient.com", "vitae@recipient.com", "laoreet@recipient.com", "urna@recipient.com", "egestas.test@recipient.com", "fusce@recipient.com", "congue@recipient.com", "nisl@recipient.com", "sagittis@recipient.com", "ligula@recipient.com", "bibendum@recipient.com", "molestie.test@recipient.com", "fusce@recipient.com", "accumsan@recipient.com", "neque@recipient.com", "a@recipient.com", "velit@recipient.com", "consequat12345@recipient.com", "id@recipient.com", "eleifend@recipient.com", "lorem@recipient.com", "congue.test@recipient.com", "nullam@recipient.com", "rhoncus@recipient.com", "quis@recipient.com", "elit@recipient.com", "eu@recipient.com", "suscipit.test@recipient.com", "nunc@recipient.com", "mollis@recipient.com", "neque@recipient.com", "eu@recipient.com", "iaculis@recipient.com", "eleifend@recipient.com", "duis@recipient.com", "vel@recipient.com", "lacus@recipient.com", "aliquet12345@recipient.com", "semper@recipient.com", "sapien@recipient.com", "sit@recipient.com",
2: CLIENT -> SERVER:    "amet12345@recipient.com", "scelerisque@recipient.com", "tortor.test@recipient.com", "duis@recipient.com", "rhoncus@recipient.com", "odio@recipient.com", "et@recipient.com", "nulla@recipient.com", "tempus12345@recipient.com", "a@recipient.com", "tristique@recipient.com", "sem@recipient.com", "consequat@recipient.com", "maecenas@recipient.com", "quis@recipient.com", "pulvinar@recipient.com", "ante@recipient.com", "sed@recipient.com", "vestibulum@recipient.com", "laoreet@recipient.com", "placerat.test@recipient.com", "mauris@recipient.com", "non@recipient.com", "enim@recipient.com", "eu@recipient.com", "mauris@recipient.com", "pretium@recipient.com", "aliquet.test@recipient.com", "ut@recipient.com", "eget@recipient.com", "vulputate@recipient.com", "urna.test@recipient.com", "fusce@recipient.com", "ut@recipient.com", "urna@recipient.com", "at@recipient.com", "augue@recipient.com", "vehicula@recipient.com", "tempus@recipient.com", "ut@recipient.com", "ut@recipient.com", "magna.test@recipient.com", "vestibulum@recipient.com",
2: CLIENT -> SERVER:    "ante@recipient.com", "ipsum@recipient.com", "primis@recipient.com", "in@recipient.com", "faucibus@recipient.com", "orci@recipient.com", "luctus@recipient.com", "et@recipient.com", "ultrices@recipient.com", "posuere@recipient.com", "cubilia@recipient.com", "curae@recipient.com", "nunc@recipient.com", "fringilla@recipient.com", "nisi@recipient.com", "in@recipient.com", "ipsum@recipient.com", "pharetra@recipient.com", "tincidunt.test@recipient.com", "cras@recipient.com", "sapien@recipient.com", "urna12345@recipient.com", "efficitur@recipient.com", "sed@recipient.com", "tempus@recipient.com", "nec12345@recipient.com", "auctor@recipient.com", "ac@recipient.com", "nibh.test@recipient.com", "pellentesque@recipient.com", "nec@recipient.com", "volutpat@recipient.com", "dui12345@recipient.com", "et@recipient.com", "faucibus@recipient.com", "lectus@recipient.com", "morbi@recipient.com", "sodales12345@recipient.com", "sapien@recipient.com", "non@recipient.com", "convallis@recipient.com", "tempor12345@recipient.com",
2: CLIENT -> SERVER:    "enim@recipient.com", "lectus@recipient.com", "aliquet@recipient.com", "lectus12345@recipient.com", "at@recipient.com", "egestas@recipient.com", "sem@recipient.com", "nunc@recipient.com", "ut@recipient.com", "pop@recipient.com", "hey@maartenmachiels.be", "pop@recipient.com"]}
2: CLIENT -> SERVER: MIME-Version: 1.0
2: CLIENT -> SERVER: Content-Type: multipart/alternative;
2: CLIENT -> SERVER:    boundary="b1_07e25a3daab58851cf911860d31022b0"
2: CLIENT -> SERVER: Content-Transfer-Encoding: 8bit
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: This is a multi-part message in MIME format.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: --b1_07e25a3daab58851cf911860d31022b0
2: CLIENT -> SERVER: Content-Type: text/plain; charset=us-ascii
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: FeunzLorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla
2: CLIENT -> SERVER: placerat, arcu quis tempus vestibulum, felis tellus tempus arcu, vel
2: CLIENT -> SERVER: mollis ex est a elit. Sed a finibus nisi, ultrices mollis ante.
2: CLIENT -> SERVER: Suspendisse porta ante vitae quam posuere, vitae laoreet urna egestas.
2: CLIENT -> SERVER: Fusce congue nisl sagittis ligula bibendum molestie. Fusce accumsan
2: CLIENT -> SERVER: neque a velit consequat, id eleifend lorem congue. Nullam rhoncus quis
2: CLIENT -> SERVER: elit eu suscipit. Nunc mollis neque eu iaculis eleifend. Duis vel
2: CLIENT -> SERVER: lacus aliquet, semper sapien sit amet, scelerisque tortor.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: Duis rhoncus odio et nulla tempus, a tristique sem consequat. Maecenas
2: CLIENT -> SERVER: quis pulvinar ante. Sed vestibulum laoreet placerat. Mauris non enim
2: CLIENT -> SERVER: eu mauris pretium aliquet. Ut eget vulputate urna. Fusce ut urna at
2: CLIENT -> SERVER: augue vehicula tempus ut ut magna. Vestibulum ante ipsum primis in
2: CLIENT -> SERVER: faucibus orci luctus et ultrices posuere cubilia Curae; Nunc fringilla
2: CLIENT -> SERVER: nisi in ipsum pharetra tincidunt. Cras sapien urna, efficitur sed
2: CLIENT -> SERVER: tempus nec, auctor ac nibh. Pellentesque nec volutpat dui, et faucibus
2: CLIENT -> SERVER: lectus. Morbi sodales, sapien non convallis tempor, enim lectus
2: CLIENT -> SERVER: aliquet lectus, at egestas sem nunc ut urna.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: Aenean in nunc faucibus, varius mi eu, sollicitudin sapien. Maecenas
2: CLIENT -> SERVER: ac volutpat nunc. Aenean sagittis faucibus odio, vitae tempor odio
2: CLIENT -> SERVER: dictum nec. Sed eget justo a erat sagittis volutpat. Aenean ut enim
2: CLIENT -> SERVER: elementum, semper tellus eget, feugiat enim. Phasellus sodales dolor
2: CLIENT -> SERVER: risus, sed tincidunt ligula congue in. Morbi congue turpis nec mi
2: CLIENT -> SERVER: tempor, eu convallis odio sagittis. Aliquam tristique efficitur neque,
2: CLIENT -> SERVER: sed rhoncus erat varius ac. Duis ac magna aliquam, iaculis diam et,
2: CLIENT -> SERVER: lacinia enim. Phasellus laoreet massa sapien, et imperdiet libero
2: CLIENT -> SERVER: bibendum a. Nullam mattis massa non nisl ultrices auctor eget eget
2: CLIENT -> SERVER: ipsum.
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: In et mauris non eros congue tincidunt. Vestibulum velit urna, rhoncus
2: CLIENT -> SERVER: et ante et, congue tempor lorem. Nulla aliquam augue non lorem semper,
2: CLIENT -> SERVER: ac faucibus ante eleifend. Sed eget tempus ex. Fusce ac fermentum est.
2: CLIENT -> SERVER: In commodo, purus non varius iaculis, tortor est imperdiet sem, non
2: CLIENT -> SERVER: semper nibh libero a urna. Phasellus ac ante a nunc condimentum rutrum
2: CLIENT -> SERVER: eu at risus. Curabitur tempus lacinia tempus. Pellentesque posuere
2: CLIENT -> SERVER: maximus diam nec mattis. Nulla pretium mollis nibh consequat dapibus.
2: CLIENT -> SERVER: Suspendisse quis libero ullamcorper, faucibus risus eu, fermentum
2: CLIENT -> SERVER: lorem. Donec lacinia ligula velit, eget accumsan neque lacinia eget.
2: CLIENT -> SERVER: Morbi vitae augue scelerisque, condimentum ligula nec, vestibulum
2: CLIENT -> SERVER: massa. Donec ornare nisi eget urna pretium sollicitudin. Morbi vitae
2: CLIENT -> SERVER: metus eu mauris elementum posuere nec nec diam.
2: CLIENT -> SERVER: Met vriendelijke groeten - Cordialement - Kind regards Challenge
2: CLIENT -> SERVER: MCWEB: http://planet.website.bePHONE: 03 650 09 80MAIL:
2: CLIENT -> SERVER: hello@website.beADDRESS: Ter Borcht 46 - 2930 Brasschaat
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: --b1_07e25a3daab58851cf911860d31022b0
2: CLIENT -> SERVER: Content-Type: text/html; charset=us-ascii
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <h1 style="font-family: Arial, Helvetica, sans-serif; font-size: 24px; line-height: 24px; margin-bottom: 20px;">Feunz</h1><p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla placerat, arcu quis tempus vestibulum, felis tellus tempus arcu, vel mollis ex est a elit. Sed a finibus nisi, ultrices mollis ante. Suspendisse porta ante vitae quam posuere, vitae laoreet urna egestas. Fusce congue nisl sagittis ligula bibendum molestie. Fusce accumsan neque a velit consequat, id eleifend lorem congue. Nullam rhoncus quis elit eu suscipit. Nunc mollis neque eu iaculis eleifend. Duis vel lacus aliquet, semper sapien sit amet, scelerisque tortor.</p>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <p>Duis rhoncus odio et nulla tempus, a tristique sem consequat. Maecenas quis pulvinar ante. Sed vestibulum laoreet placerat. Mauris non enim eu mauris pretium aliquet. Ut eget vulputate urna. Fusce ut urna at augue vehicula tempus ut ut magna. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Nunc fringilla nisi in ipsum pharetra tincidunt. Cras sapien urna, efficitur sed tempus nec, auctor ac nibh. Pellentesque nec volutpat dui, et faucibus lectus. Morbi sodales, sapien non convallis tempor, enim lectus aliquet lectus, at egestas sem nunc ut urna.</p>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <p>Aenean in nunc faucibus, varius mi eu, sollicitudin sapien. Maecenas ac volutpat nunc. Aenean sagittis faucibus odio, vitae tempor odio dictum nec. Sed eget justo a erat sagittis volutpat. Aenean ut enim elementum, semper tellus eget, feugiat enim. Phasellus sodales dolor risus, sed tincidunt ligula congue in. Morbi congue turpis nec mi tempor, eu convallis odio sagittis. Aliquam tristique efficitur neque, sed rhoncus erat varius ac. Duis ac magna aliquam, iaculis diam et, lacinia enim. Phasellus laoreet massa sapien, et imperdiet libero bibendum a. Nullam mattis massa non nisl ultrices auctor eget eget ipsum.</p>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: <p>In et mauris non eros congue tincidunt. Vestibulum velit urna, rhoncus et ante et, congue tempor lorem. Nulla aliquam augue non lorem semper, ac faucibus ante eleifend. Sed eget tempus ex. Fusce ac fermentum est. In commodo, purus non varius iaculis, tortor est imperdiet sem, non semper nibh libero a urna. Phasellus ac ante a nunc condimentum rutrum eu at risus. Curabitur tempus lacinia tempus. Pellentesque posuere maximus diam nec mattis. Nulla pretium mollis nibh consequat dapibus. Suspendisse quis libero ullamcorper, faucibus risus eu, fermentum lorem. Donec lacinia ligula velit, eget accumsan neque lacinia eget. Morbi vitae augue scelerisque, condimentum ligula nec, vestibulum massa. Donec ornare nisi eget urna pretium sollicitudin. Morbi vitae metus eu mauris elementum posuere nec nec diam.</p>
2: CLIENT -> SERVER: <p style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; line-height: 20px; margin-bottom: 20px;">Met vriendelijke groeten - Cordialement - Kind regards <br /></p><table style="border: none; font-family: Arial, Helvetica, sans-serif; font-size: 12px; line-height: 20px; margin-bottom: 20px;"><tr><td style="padding-right: 20px;"><img src="http://planet.website.be/img/logo.png" alt=" logo" /></td><td>WEB: <a style="color: #CA0A42;" href="http://planet.website.be" title="Visit  Planet">http://planet.website.be</a><br />PHONE: <a style="color: #CA0A42;" href="tel:036500980" class="cicon telephone">03 55555550</a><br />MAIL: <a style="color: #CA0A42;" href="mailto:hello@website.be" class="cicon email">hello@website.be</a><br />ADDRESS: <span class="cicon address">Address</span></td></tr></table>
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: --b1_07e25a3daab58851cf911860d31022b0--
2: CLIENT -> SERVER: 
2: CLIENT -> SERVER: .
2: SERVER -> CLIENT: 250 Ok: queued as hAnkBkiFRAKc9T0oDDQRlw
2: CLIENT -> SERVER: QUIT
2: SERVER -> CLIENT: 221 See you later

When I send it like this, I get no invalid e-mail notifications. No wrapping in between e-mail addresses... Any remarks?

The new log in TXT for your convenience. newlog.txt

[Synchro]

That extra char won't doing anything because you've escaped it first, so it's just ASCII and thus doesn't trigger the alternative encoding mechanism. Put it in as raw UTF-8 and let PHPMailer encode it, like "a":"é", not "a":"\u00e9". (Oh, and set $mail->CharSet = 'utf-8' as well to ensure the charset matches).

[chocolata]

Hi, sorry, I haven't escaped it :-) I edited my response. This is what I did:

    $sendgrid_options = array(  'category'=>array(g::site_root,$send_type),
        'unique_args'=> array('token'=>$token,'from_url'=>g::site_root,"a"=>"é"));

A little further down:

        $mail->isHTML(true);
        $mail->WordWrap = 70;
        $mail->CharSet = 'UTF-8';

        // Set headers
        $mail->setFrom($from_address, $from_name);
        $mail->addReplyTo($reply_address, $reply_name);
        $mail->addAddress($to_address, $to_name);
        $mail->Subject = $subject;

        // Some custom encoding, to avoid sendgrid issues!
        $sendgrid_options = str_replace(',', ', ',json_encode($sendgrid_options));
        $mail->addCustomHeader('x-smtpapi', $mail->encodeHeader($sendgrid_options)); // Metadata + sender info

        // Send it
        $mail->msgHTML($html_message);
        if (!$mail->Send()) { ... }

[Synchro]

Hm. json_encode shouldn't be doing that since it requires you to use UTF-8 to start with - check what's in the $sendgrid_options string before giving it to PHPMailer.

You really should not call encodeHeader yourself; PHPMailer does it for you when it builds the message, and calling it twice on the same header is unlikely to do anything good.

[Synchro]

Found the cause - you need to set the JSON_UNESCAPED_UNICODE option on json_encode.

[chocolata]

Hi, thanks! I now removed the encodeHeader call in my custom header. Now, the mails seem to arrive, but the log is looks a bit messed up...

This is the part with the XSMTPAPI header. Is this good or bad? :-$

2: CLIENT -> SERVER: x-smtpapi: =?UTF-8?Q?{"category":["https:\/\/cmcrm.websiteee.be",_"wysiwyg"],_"uniqu?=
2: CLIENT -> SERVER:  =?UTF-8?Q?e=5Fargs":{"token":"5743002dcfd6c",_"from=5Furl":"https:\/\/cmc?=
2: CLIENT -> SERVER:  =?UTF-8?Q?rm.websiteee.be",_"a":"=C3=A9"},_"to":["lorem@websiteee.be",_"i?=
2: CLIENT -> SERVER:  =?UTF-8?Q?psum@websiteee.be",_"dolor@websiteee.be",_"sit@websiteee.be",_"?=
2: CLIENT -> SERVER:  =?UTF-8?Q?amet12345@websiteee.be",_"consectetur@websiteee.be",_"adipiscin?=
2: CLIENT -> SERVER:  =?UTF-8?Q?g@websiteee.be",_"elit.test@websiteee.be",_"nulla@websiteee.be"?=
2: CLIENT -> SERVER:  =?UTF-8?Q?,_"placerat12345@websiteee.be",_"arcu@websiteee.be",_"quis@choc?=
2: CLIENT -> SERVER:  =?UTF-8?Q?olata.be",_"tempus@websiteee.be",_"vestibulum12345@websiteee.be?=
2: CLIENT -> SERVER:  =?UTF-8?Q?",_"felis@websiteee.be",_"tellus@websiteee.be",_"tempus@chocola?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ta.be",_"arcu12345@websiteee.be",_"vel@websiteee.be",_"mollis@c?=
2: CLIENT -> SERVER:  =?UTF-8?Q?hocolata.be",_"ex@websiteee.be",_"est@websiteee.be",_"a@chocola?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ta.be",_"elit.test@websiteee.be",_"sed@websiteee.be",_"a@chocol?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ata.be",_"finibus@websiteee.be",_"nisi12345@websiteee.be",_"ult?=
2: CLIENT -> SERVER:  =?UTF-8?Q?rices@websiteee.be",_"mollis@websiteee.be",_"ante.test@chocolat?=
2: CLIENT -> SERVER:  =?UTF-8?Q?a.be",_"suspendisse@websiteee.be",_"porta@websiteee.be",_"ante@?=
2: CLIENT -> SERVER:  =?UTF-8?Q?websiteee.be",_"vitae@websiteee.be",_"quam@websiteee.be",_"posu?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ere12345@websiteee.be",_"vitae@websiteee.be",_"laoreet@chocolat?=
2: CLIENT -> SERVER:  =?UTF-8?Q?a.be",_"urna@websiteee.be",_"egestas.test@websiteee.be",_"fusce?=
2: CLIENT -> SERVER:  =?UTF-8?Q?@websiteee.be",_"congue@websiteee.be",_"nisl@websiteee.be",_"sa?=
2: CLIENT -> SERVER:  =?UTF-8?Q?gittis@websiteee.be",_"ligula@websiteee.be",_"bibendum@chocolat?=
2: CLIENT -> SERVER:  =?UTF-8?Q?a.be",_"molestie.test@websiteee.be",_"fusce@websiteee.be",_"acc?=
2: CLIENT -> SERVER:  =?UTF-8?Q?umsan@websiteee.be",_"neque@websiteee.be",_"a@websiteee.be",_"v?=
2: CLIENT -> SERVER:  =?UTF-8?Q?elit@websiteee.be",_"consequat12345@websiteee.be",_"id@chocolat?=
2: CLIENT -> SERVER:  =?UTF-8?Q?a.be",_"eleifend@websiteee.be",_"lorem@websiteee.be",_"congue.t?=
2: CLIENT -> SERVER:  =?UTF-8?Q?est@websiteee.be",_"nullam@websiteee.be",_"rhoncus@websiteee.be?=
2: CLIENT -> SERVER:  =?UTF-8?Q?",_"quis@websiteee.be",_"elit@websiteee.be",_"eu@websiteee.be",?=
2: CLIENT -> SERVER:  =?UTF-8?Q?_"suscipit.test@websiteee.be",_"nunc@websiteee.be",_"mollis@cho?=
2: CLIENT -> SERVER:  =?UTF-8?Q?colata.be",_"neque@websiteee.be",_"eu@websiteee.be",_"iaculis@c?=
2: CLIENT -> SERVER:  =?UTF-8?Q?hocolata.be",_"eleifend@websiteee.be",_"duis@websiteee.be",_"ve?=
2: CLIENT -> SERVER:  =?UTF-8?Q?l@websiteee.be",_"lacus@websiteee.be",_"aliquet12345@websiteee.?=
2: CLIENT -> SERVER:  =?UTF-8?Q?be",_"semper@websiteee.be",_"sapien@websiteee.be",_"sit@chocola?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ta.be",_"amet12345@websiteee.be",_"scelerisque@websiteee.be",_"?=
2: CLIENT -> SERVER:  =?UTF-8?Q?tortor.test@websiteee.be",_"duis@websiteee.be",_"rhoncus@chocol?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ata.be",_"odio@websiteee.be",_"et@websiteee.be",_"nulla@chocola?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ta.be",_"tempus12345@websiteee.be",_"a@websiteee.be",_"tristiqu?=
2: CLIENT -> SERVER:  =?UTF-8?Q?e@websiteee.be",_"sem@websiteee.be",_"consequat@websiteee.be",_?=
2: CLIENT -> SERVER:  =?UTF-8?Q?"maecenas@websiteee.be",_"quis@websiteee.be",_"pulvinar@chocola?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ta.be",_"ante@websiteee.be",_"sed@websiteee.be",_"vestibulum@ch?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ocolata.be",_"laoreet@websiteee.be",_"placerat.test@websiteee.b?=
2: CLIENT -> SERVER:  =?UTF-8?Q?e",_"mauris@websiteee.be",_"non@websiteee.be",_"enim@websiteee.?=
2: CLIENT -> SERVER:  =?UTF-8?Q?be",_"eu@websiteee.be",_"mauris@websiteee.be",_"pretium@chocola?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ta.be",_"aliquet.test@websiteee.be",_"ut@websiteee.be",_"eget@c?=
2: CLIENT -> SERVER:  =?UTF-8?Q?hocolata.be",_"vulputate@websiteee.be",_"urna.test@websiteee.be?=
2: CLIENT -> SERVER:  =?UTF-8?Q?",_"fusce@websiteee.be",_"ut@websiteee.be",_"urna@websiteee.be"?=
2: CLIENT -> SERVER:  =?UTF-8?Q?,_"at@websiteee.be",_"augue@websiteee.be",_"vehicula@websiteee.?=
2: CLIENT -> SERVER:  =?UTF-8?Q?be",_"tempus@websiteee.be",_"ut@websiteee.be",_"ut@websiteee.be?=
2: CLIENT -> SERVER:  =?UTF-8?Q?",_"magna.test@websiteee.be",_"vestibulum@websiteee.be",_"ante@?=
2: CLIENT -> SERVER:  =?UTF-8?Q?websiteee.be",_"ipsum@websiteee.be",_"primis@websiteee.be",_"in?=
2: CLIENT -> SERVER:  =?UTF-8?Q?@websiteee.be",_"faucibus@websiteee.be",_"orci@websiteee.be",_"?=
2: CLIENT -> SERVER:  =?UTF-8?Q?luctus@websiteee.be",_"et@websiteee.be",_"ultrices@websiteee.be?=
2: CLIENT -> SERVER:  =?UTF-8?Q?",_"posuere@websiteee.be",_"cubilia@websiteee.be",_"curae@choco?=
2: CLIENT -> SERVER:  =?UTF-8?Q?lata.be",_"nunc@websiteee.be",_"fringilla@websiteee.be",_"nisi@?=
2: CLIENT -> SERVER:  =?UTF-8?Q?websiteee.be",_"in@websiteee.be",_"ipsum@websiteee.be",_"pharet?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ra@websiteee.be",_"tincidunt.test@websiteee.be",_"cras@chocolat?=
2: CLIENT -> SERVER:  =?UTF-8?Q?a.be",_"sapien@websiteee.be",_"urna12345@websiteee.be",_"effici?=
2: CLIENT -> SERVER:  =?UTF-8?Q?tur@websiteee.be",_"sed@websiteee.be",_"tempus@websiteee.be",_"?=
2: CLIENT -> SERVER:  =?UTF-8?Q?nec12345@websiteee.be",_"auctor@websiteee.be",_"ac@websiteee.be?=
2: CLIENT -> SERVER:  =?UTF-8?Q?",_"nibh.test@websiteee.be",_"pellentesque@websiteee.be",_"nec@?=
2: CLIENT -> SERVER:  =?UTF-8?Q?websiteee.be",_"volutpat@websiteee.be",_"dui12345@websiteee.be"?=
2: CLIENT -> SERVER:  =?UTF-8?Q?,_"et@websiteee.be",_"faucibus@websiteee.be",_"lectus@websiteee?=
2: CLIENT -> SERVER:  =?UTF-8?Q?.be",_"morbi@websiteee.be",_"sodales12345@websiteee.be",_"sapie?=
2: CLIENT -> SERVER:  =?UTF-8?Q?n@websiteee.be",_"non@websiteee.be",_"convallis@websiteee.be",_?=
2: CLIENT -> SERVER:  =?UTF-8?Q?"tempor12345@websiteee.be",_"enim@websiteee.be",_"lectus@chocol?=
2: CLIENT -> SERVER:  =?UTF-8?Q?ata.be",_"aliquet@websiteee.be",_"lectus12345@websiteee.be",_"a?=
2: CLIENT -> SERVER:  =?UTF-8?Q?t@websiteee.be",_"egestas@websiteee.be",_"sem@websiteee.be",_"n?=
2: CLIENT -> SERVER:  =?UTF-8?Q?unc@websiteee.be",_"ut@websiteee.be",_"pop@websiteee.be",_"hey@?=
2: CLIENT -> SERVER:  =?UTF-8?Q?maartenmachiels.be",_"pop@websiteee.be",_"maartenmachiels@gmail?=
2: CLIENT -> SERVER:  =?UTF-8?Q?.com"]}?=

This is the part now that sets the XSMTPAPI header:

        // Force custom encoding, to avoid sendgrid issues!
        $sendgrid_options = str_replace(',', ', ',json_encode($sendgrid_options, JSON_UNESCAPED_UNICODE));
        $mail->addCustomHeader('x-smtpapi', $sendgrid_options); // Metadata + sender info

Do I still need the {"a":"é"} thing in my XSMTPAPI header? :-)

[Synchro]

Yep, that looks fine - when I said it "looks quite different", that's what I meant!

What's going on here is that email headers only support ASCII, and if your content is ASCII (7-bit compatible), it just needs folding and that's all. As soon as you have any 8-bit chars, it can't do that and so it has to use Q or B encoding to make the header ASCII-safe. They're effectively inline versions of quoted-printable and base64 encoding respectively. By including a single 8-bit character, we're forcing it to use Q encoding, which is more efficient than B encoding if less than 1/3 of the chars need encoding.

So, yes, you do need to keep that extra char in to force Q encoding. This is really down to sendgrid - I can't see anything amiss with our normally-encoded header (and I'd like to know if there is!), so we're only having to do that to work around the problem at their end, since they seem to be able to handle Q-encoding correctly

The folding mechanism is exactly the same in all of them.

[chocolata]

Thanks Synchro for your follow-up. It seems this issue is now resolved. I haven't heard back from SendGrid support yet, but when I do, I'll be sure to direct them to this thread. If I get any follow-up info from them, I'll be sure to post it here.

One last tweak I did was to remove the str_replace() that adds extra spaces in the XSMTPAPI header. I thought the folding might have had something to do with that, but after my final testing, it doesn't. As you say, the problem is in the encoding, so I can safely remove it.

Thanks for your efforts. My problem is solved!

[Synchro]

Glad it worked - but it will be good to get to the bottom of it!

[Luisinho77]

Yo!

What a shame, I wish things could happen differently.. Just take a look here http://physhynunke.leggumms.com/e4bsi

Best wishes, ljlorente

[Luisinho77]

Hi friend!

I've got some really great things from that store, they are so cool, just take a look http://positive.mydancefitstudio.com/e4yhlb

Best, ljlorente