Closed 7i77an closed 4 years ago
7i77an
commented
4 years ago Vincent,
Check this:
https://tools.ietf.org/html/rfc2047#section-8
(=?ISO-8859-1?Q?a?= =?ISO-8859-2?Q?_b?=) (a b)
In order to cause a SPACE to be displayed between two strings
of encoded text, the SPACE MAY be encoded as part of one of
the 'encoded-word's.
vincent-richard
commented
4 years ago Hello!
I don't think there is an issue there, at least, not with VMime. Given the following test:
vmime::text t;
t.parse("=?utf-8?Q?Adquisi=C3=B3n?= de Laptop y celular");
VASSERT_EQ("parse", "Adquisión de Laptop y celular", t.getWholeBuffer());
vmime::text t("Adquisión de Laptop y celular", vmime::charset("UTF-8"));
VASSERT_EQ("generate", "=?UTF-8?Q?Adquisi=C3=B3n?= de Laptop y celular", t.generate());It seems there is no issue with encoding or decoding the whitespace, VMime follows the RFC and generate perfectly valid encoded text.
Could you please send a SSCCE that exhibits the problem? (not involving connection to a server) I suspect there might something else that causes the issue, and the log is not clear about this. And I don't even see the relation with SSL/TLS.
What are the bytes sent by VMime to the SMTP(S) server? There can be a difference between what is sent to the server, and what is received by the client, something might be altered.
AFAIK, the following subject is totally valid:
Subject: =?utf-8?Q?Adquisi=C3=B3n?= de Laptop y celularOn decoding, the space between the encoded word and the unencoded text following MUST be displayed by a client, and does not need to be encoded.
This only applies to a space between two encoded words, which is not the case here.
By the way, in section 8, there is this precise case:
CC: =?ISO-8859-1?Q?Andr=E9?= Pirard PIRARD@vm1.ulg.ac.be
where it should decode to something like "André Pirard", with the space between the 2 words (first one encoded, second one not).
I need more details.
vincent-richard
commented
4 years ago Also, be sure to test with the latest code, from the repository, not a release.
7i77an
commented
4 years ago Hi,
I send email from hotmail account and receive this into thunderbird client (client doesn't matter, in other clients the problem also happens)
Fields data are: From: "André Pirard XXXXX@hotmail.com" Subject: ""Nueva Adquisión de Laptop y celula""
Checking email source:
Subject: Nueva Adquisiónde Laptop y celula From: =?utf-8?Q?Andr=C3=A9?= =?us-ascii?Q?Pirard?= XXXXX@hotmail.com To: XXXXX@XXXXX.com Date: Wed, 10 Jun 2020 11:33:29 -0300 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64
Both fields are displayed poorly: From: "AndréPirard XXXXX@hotmail.com" Subject: "Nueva Adquisiónde Laptop y celula"
You can try this yourself by sending from a hotmail account?? If you don't have a hotmail account, I can provide you with the credentials for this test account.
The authentication is with username and password and smtp port is 587 (TLS)
Thanks Vincent.
7i77an
commented
4 years ago Vincent,
I change static SendMessage from example6.cpp
static void sendMessage() {
try {
// Request user to enter an URL
std::cout << "Enter an URL to connect to transport service." << std::endl;
std::cout << "Available protocols: " << findAvailableProtocols(vmime::net::service::TYPE_TRANSPORT) << std::endl;
std::cout << "(eg. smtp://myserver.com, sendmail://localhost)" << std::endl;
std::cout << "> ";
std::cout.flush();
vmime::messageBuilder mb;
vmime::string urlString;
std::getline(std::cin, urlString);
vmime::utility::url url(urlString);
url.setUsername("XXXXXXX@hotmail.com");
url.setPassword("XXXXXXX");
vmime::shared_ptr <vmime::net::transport> tr;
if (url.getUsername().empty() || url.getPassword().empty()) {
tr = g_session->getTransport(url, vmime::make_shared <interactiveAuthenticator>());
} else {
tr = g_session->getTransport(url);
} // Enable TLS support if available
tr->setProperty("connection.tls", true);
// Set the time out handler
tr->setTimeoutHandlerFactory(vmime::make_shared <timeoutHandlerFactory>());
// Set the object responsible for verifying certificates, in the
// case a secured connection is used (TLS/SSL)
tr->setCertificateVerifier(
vmime::make_shared <interactiveCertificateVerifier>()
); // You can also set some properties (see example7 to know the properties
// available for each service). For example, for SMTP:
if (!url.getUsername().empty() || !url.getPassword().empty()) {
tr->setProperty("options.need-authentication", true);
}
// Trace communication between client and server
vmime::shared_ptr <std::ostringstream> traceStream = vmime::make_shared <std::ostringstream>();
tr->setTracerFactory(vmime::make_shared <myTracerFactory>(traceStream));
// Information about the mail
std::cout << "Enter email of the expeditor (eg. me@somewhere.com): ";
std::cout.flush();
vmime::string fromString;
std::getline(std::cin, fromString);
vmime::mailbox from(fromString);
vmime::mailboxList to;
for (bool cont = true ; cont ; ) {
std::cout << "Enter email of the recipient (empty to stop): ";
std::cout.flush();
vmime::string toString;
std::getline(std::cin, toString);
cont = (toString.size() != 0);
if (cont) {
mb.getRecipients().appendAddress(vmime::make_shared <vmime::mailbox>(toString));
}
}
std::cout << "Enter message data, including headers (end with '.' on a single line):" << std::endl;
std::ostringstream data;
for (bool cont = true ; cont ; ) {
std::string line;
std::getline(std::cin, line);
if (line == ".") {
cont = false;
} else {
data << line << "\r\n";
}
}
mb.setExpeditor(from);
vmime::string subject = "Andr� Pirard";
mb.setSubject(vmime::text(subject, vmime::charsets::UTF_8));
//Construyo el mensaje a enviar
vmime::shared_ptr<vmime::message> msg = mb.construct();
// Connect to server
tr->connect();
// Send the message// vmime::string msgData = data.str(); // vmime::utility::inputStreamStringAdapter vis(msgData);
// tr->send(from, to, vis, msgData.length()); tr->send(msg);
// Display connection log
std::cout << std::endl;
std::cout << "Connection Trace:" << std::endl;
std::cout << "=================" << std::endl;
std::cout << traceStream->str();
tr->disconnect();
} catch (vmime::exception& e) {
std::cerr << std::endl;
std::cerr << e << std::endl;
throw;
} catch (std::exception& e) {
std::cerr << std::endl;
std::cerr << "std::exception: " << e.what() << std::endl;
throw;
}}
Input data: url: smtp://smtp-mail.outlook.com from: XXXXX@hotmail.com to: XXXXXX
When I receive the email, the subject is displayed incorrectly: Subject: AndréPirard
If you don't have a hotmail account, I can provide you with the credentials for this test account.
vincent-richard
commented
4 years ago I will try to make a test on my own, I have an old Hotmail account. But, meanwhile, it would save me some time if you could send me:
1) the headers from the message immediately generated by VMime (just before sending it to the SMTP server) 2) the headers from the received message in your Hotmail client (after it was received, and maybe altered, by the SMTP server)
This is not very clear in your messages, when you are talking about "email source".
7i77an
commented
4 years ago Vincent,
I send you the headers of 2 tests that I did
[smtp:1] C: Connecting to smtp-mail.outlook.com, port 25
[smtp:1] S: 220 CP2PR80CA0007.outlook.office365.com Microsoft ESMTP MAIL Service ready at Thu, 11 Jun 2020 22:20:07 +0000
[smtp:1] C: EHLO jessie64-builder-2
[smtp:1] S: 250-CP2PR80CA0007.outlook.office365.com Hello [190.64.140.66]
[smtp:1] S: 250-SIZE 157286400
[smtp:1] S: 250-PIPELINING
[smtp:1] S: 250-DSN
[smtp:1] S: 250-ENHANCEDSTATUSCODES
[smtp:1] S: 250-STARTTLS
[smtp:1] S: 250-8BITMIME
[smtp:1] S: 250-BINARYMIME
[smtp:1] S: 250-CHUNKING
[smtp:1] S: 250 SMTPUTF8
[smtp:1] C: STARTTLS
[smtp:1] S: 220 2.0.0 SMTP server ready
[smtp:1] C: Beginning SSL/TLS handshake
[smtp:1] C: Getting peer certificates
[smtp:1] C: EHLO jessie64-builder-2
[smtp:1] S: 250-CP2PR80CA0007.outlook.office365.com Hello [190.64.140.66]
[smtp:1] S: 250-SIZE 157286400
[smtp:1] S: 250-PIPELINING
[smtp:1] S: 250-DSN
[smtp:1] S: 250-ENHANCEDSTATUSCODES
[smtp:1] S: 250-AUTH LOGIN XOAUTH2
[smtp:1] S: 250-8BITMIME
[smtp:1] S: 250-BINARYMIME
[smtp:1] S: 250-CHUNKING
[smtp:1] S: 250 SMTPUTF8
[smtp:1] C: AUTH LOGIN
[smtp:1] S: 334 VXNlcm5hbWU6
[smtp:1] C: {...SASL exchange: 44 bytes of data...}
[smtp:1] S: 334 UGFzc3dvcmQ6
[smtp:1] C: {...SASL exchange: 16 bytes of data...}
[smtp:1] S: 235 2.7.0 Authentication successful
[smtp:1] C: MAIL FROM:XXXXX@hotmail.com SMTPUTF8 SIZE=373
[smtp:1] C: RCPT TO:
Subject: =?utf-8?Q?Andr=3F?= Pirard From: XXXXX@hotmail.com To: XXXXX Date: Thu, 11 Jun 2020 22:20:08 +0000 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: base64
From - Thu Jun 11 19:21:48 2020
X-Account-Key: account4
X-UIDL: 71051.rX,NamASTODyORMBkQPKI,4lcB3h+Rck5aO19OTr0GU=
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:
Return-Path: XXXXX@hotmail.com
Received: from zcs-XXXXX (LHLO
XXXXX) (200.40.135.117) by XXXXX with LMTP;
Thu, 11 Jun 2020 19:18:29 -0300 (UYT)
Received: from localhost (localhost [127.0.0.1])
by XXXXX (Postfix) with ESMTP id 3AA0A22D99FFC
for
Delivered-To: XXXXX@gmail.com Received: by 2002:a2e:844f:0:0:0:0:0 with SMTP id u15csp22183ljh; Thu, 11 Jun 2020 15:49:18 -0700 (PDT) X-Received: by 2002:a67:2d16:: with SMTP id t22mr9092766vst.160.1591915757851; Thu, 11 Jun 2020 15:49:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591915757; cv=none; d=google.com; s=arc-20160816; b=Z9V8IU1xy2qY8P6y4qDNYDFnL9jpPdgAcUe9+zjYvkl5PdhrzTU8Zkr+LplOVFc2fy lneaPyGSnYLtbvAdHu/as8leDjDHLlwGkDBpJuFsyZABinQlrRTbifygLl7iCRE1F/SN 8wuZdHs1XZj06voRweWTbBmbYPA/7dUIoAh50S6Yr3pesINXa/0E3YvtyENbrG0pUhi8 RNIX/mJ266IgnRctIiXaB17RcLZt2ll+O3/MN1agjv1eFPYe+/7X9wHbCXgThQ0kRvS3 RHPcLlyTTMJF6hWLL+1cCSitHnEm4ffr5NW2OIq54bpMLK8Z4MFDUkAxObcjBYEpaYNq FAEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:mime-version:date:to:from:subject:dkim-signature; bh=nFRfbl7X8OD8baqAU7BOqvkjk3RhZoZeGDDs/IGsf8E=; b=URVgp9KTvH+kn1uw9JXlVL6Nt//vxaB0QoOmFJOA+nYeJfT1esLqtI09Y5HWZluc/u x6PIkBimzIMwh7j84Sl9rdTJqV9CV+4XT26V2ijZDcrcwDUeTaUNoBcLdDJ8foXG0rSJ gVD5JXcLdqtESMgf4hKevfXz5mqMh3zZuMiySkYPrFf1oDbRqiTKqLgvXroG8+ilqRcr 9PDJpjy7y0F1bHsg3M54TsC2MYEYvwZuNMx9BR9TA7MHAeswfVSzgevuwQ+pS3PWimHU B9viKcTSR48dxw+H0oydGRatYH4WsTWejDU/q4DasdZeBpQuZwMpAnQWRELb76d+OYTq sIRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@XXXXX header.s=google header.b=e73vqqet; spf=pass (google.com: domain of servicedesk@XXXXX designates 209.85.220.41 as permitted sender) smtp.mailfrom=servicedesk@XXXXX Return-Path: servicedesk@XXXXX Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id p187sor2759614vsa.20.2020.06.11.15.49.17 for XXXXX@gmail.com (Google Transport Security); Thu, 11 Jun 2020 15:49:17 -0700 (PDT) Received-SPF: pass (google.com: domain of servicedesk@XXXXX designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@XXXXX header.s=google header.b=e73vqqet; spf=pass (google.com: domain of servicedesk@XXXXX designates 209.85.220.41 as permitted sender) smtp.mailfrom=servicedesk@XXXXX DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=XXXXX; s=google; h=subject:from:to:date:mime-version:message-id; bh=nFRfbl7X8OD8baqAU7BOqvkjk3RhZoZeGDDs/IGsf8E=; b=e73vqqetXy8vxoUbLKgDj3XojKlj41kuDcgsyGhiaW9lc0bIk9KDdKlSajgum/LSvh lZHga0bvrpWO3rnH8iNwWrKZhhsqb4CrHdUzFjZGersaBlxHi3AfL6AIznxFKvJNQ3ky MxC0RYqdtPgTri5PVqzUQM86WsJgIUQc5epOg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:date:mime-version:message-id; bh=nFRfbl7X8OD8baqAU7BOqvkjk3RhZoZeGDDs/IGsf8E=; b=CTH+Xn95yw1QfwBmISrtU54UdltSAEnsgkF/Uump7jvI+T4odq2AUFGPvhxSa/ID5y 9llRmI/sen8yyMLzdnsyxA4Ud7maBv4ZUIh6HolSHWDboabJBfGdgDqFz1s+8sY5hTBV H6RmsZH4ck2rVWTlVxJCqx0hQpajKnX2TjBfkpxQJBB320jcoldgpnsf12XNhzxRQOfk O8m4xsruc+uBbWgMjCsTDlcEb1JURJOwF7HmrU6FoI0pm6yMCXkSxBs0Eqhjcwt+n6zt 6zEbFrLQHHPWSpmerNJA27tRSznRDTLAT9nFf3dEZJTscvjQegViVu3CbVIq69nidoU4 fi9A== X-Gm-Message-State: AOAM533E+vv/Q3xvLHDSShvdc9Dw5g43W28Vzlw5mdninAWHQ50oaNrY g35cJ7VPKgU5jff1TdzcUd7l8P7ktd+ku2JiQC5ql3o8Dv2oIMXAZS1NsPuKm6XCypTgQ0GYGpE tBC0LLVz7NXNvnuC9tCZqUow651M2hOIItRjP59mKycxXQR+odxTw1nKdQOqvTGaWv3GWb3yMvQ == X-Google-Smtp-Source: ABdhPJzbSAE9+OZQU8mAk7kqWs9A+1ZGfhrFsr/OPuc4+61AqaXG4gHGbJUZ4YB1ucbf6mb1MMBg5A== X-Received: by 2002:a05:6102:50e:: with SMTP id l14mr7746077vsa.20.1591915756273; Thu, 11 Jun 2020 15:49:16 -0700 (PDT) Return-Path: servicedesk@XXXXX Received: from cls5-cgn-mia-B1082 ([38.132.207.152]) by smtp.gmail.com with UTF8SMTPSA id v18sm622844vsd.7.2020.06.11.15.49.15 for XXXXX@gmail.com (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Jun 2020 15:49:15 -0700 (PDT) Subject: Adquisiciónde Laptop From: Servicedesk servicedesk@XXXXX To: XXXXX@gmail.com Date: Thu, 11 Jun 2020 18:49:15 -0400 Mime-Version: 1.0 Message-Id: 41133431dc7043a3b2582be50d7c2bf8@inconcertcc.com Content-Type: multipart/related; boundary="=_HjHUU4zr9elowtQIw-K4w5vcxmtT8Q3ZUwJQ8iksUOzi8g5S"
--=_HjHUU4zr9elowtQIw-K4w5vcxmtT8Q3ZUwJQ8iksUOzi8g5S Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
vincent-richard
commented
4 years ago I think I finally found the issue! It is related to International Email Support, where no encoding is needed in eg. header fields.
This was quite tricky to investigate because generating the message without the "International Email" option outputs a different result. Also, International Email support is enabled for generating emails when the SMTP server supports the SMTPUTF8 extension.
Please try the following patch and let me know: http://www.vmime.org/public/download/patches/vmime-20200612-wordencintlemail.patch
7i77an
commented
4 years ago Thanks vincent!! I review it and I answer you.
7i77an
commented
4 years ago Thanks a lot vincent, the patch worked perfectly. I leave the ticket open in case you want to relate it to the commit.
vincent-richard
commented
4 years ago Thanks for your report and your feedback, I merged the fix. Closing the issue now.
Hi Vicent,
I have a similar problem #208.
mb.setSubject(vmime::text(subject, vmime::charsets::UTF_8)); subject value: "Adquisión de Laptop y celular"
After construct de msg, log this: Subject: =?utf-8?Q?Adquisi=C3=B3n?= de Laptop y celular From: =?utf-8?Q?DanielPe=C3=B1a?= pena.daniel.1976@gmail.com
The value received into field subject is: "Adquisiónde Laptop y celular". The space character is missing.
This only happens in the subject field and with the SSL/TLS protocol.
I send this email to gmail account or hotmail account and I have this problem. If I send this email to another account without SSL, the problem It does not happen.
I think but I am not sure that the problem may be in the handshake between the library and the mail server when using the SSL / TLS protocol.
I converted the msg to txt and the value of the subject field looks good. I must assume that the problem is how the mail server interprets what the library generates.
Attach profile log:
[DEBUG] 6/8-13:07:03.361354 20041 PABLO => CEMailController::SendEMail - subject 'Adquisión de Laptop y celular' [DEBUG] 6/8-13:07:03.361570 20041 CAccount::SendEMail: Sending email '0eab847df07342f298fbaa08391f1e6b@inconcertcc.com' from account MAIL_pena.daniel.1976@gmail.com [DEBUG] 6/8-13:07:03.361578 20041 PABLO => CAccount::SendEMail: Subject: Adquisión de Laptop y celular [PROFI] 6/8-13:07:03.372863 20041 EMailProxy [smtps:1] Send C:Connecting to smtp.gmail.com, port 465 [PROFI] 6/8-13:07:03.448058 20041 EMailProxy [smtps:1] Send C:Beginning SSL/TLS handshake [PROFI] 6/8-13:07:03.527290 20041 EMailProxy [smtps:1] Send C:Getting peer certificates [WARN ] 6/8-13:07:03.530875 20041 CustomCertificateVerifier::verify: Certificate for url 'smtps://smtp.gmail.com' is not valid. Anyway, the certificate has been saved to file '/etc/inconcert/ca-certs/3n2eor4Zs2cIAAAAAEIOQw==.pem as a valid certificate'. For security, check the certificate information. [INFO ] 6/8-13:07:03.531002 20041 Loading certificate from memory 3n2eor4Zs2cIAAAAAEIOQw== - OK [PROFI] 6/8-13:07:04.794497 20041 EMailProxy [smtps:1] Receive C:220 smtp.gmail.com ESMTP n1sm6713466qkf.54 - gsmtp [PROFI] 6/8-13:07:04.795083 20041 EMailProxy [smtps:1] Send C:EHLO inconcertcc [PROFI] 6/8-13:07:04.969089 20041 EMailProxy [smtps:1] Receive C:250-smtp.gmail.com at your service, [190.64.140.66] [PROFI] 6/8-13:07:04.969120 20041 EMailProxy [smtps:1] Receive C:250-SIZE 35882577 [PROFI] 6/8-13:07:04.969126 20041 EMailProxy [smtps:1] Receive C:250-8BITMIME [PROFI] 6/8-13:07:04.969135 20041 EMailProxy [smtps:1] Receive C:250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH [PROFI] 6/8-13:07:04.969139 20041 EMailProxy [smtps:1] Receive C:250-ENHANCEDSTATUSCODES [PROFI] 6/8-13:07:04.969144 20041 EMailProxy [smtps:1] Receive C:250-PIPELINING [PROFI] 6/8-13:07:04.969147 20041 EMailProxy [smtps:1] Receive C:250-CHUNKING [PROFI] 6/8-13:07:04.969151 20041 EMailProxy [smtps:1] Receive C:250 SMTPUTF8 [PROFI] 6/8-13:07:04.973580 20041 EMailProxy [smtps:1] Send C:AUTH XOAUTH2 dXNlcj1wZW5hLmRhbmllbC4xOTc2QGdtYWlsLmNvbQFhdXRoPUJlYXJlciB5YTI5LmEwQWZINlNNRGNJLU1lTk15TENvSUxDdENWVGctSmNCS3JCbXo0ckhhMkp5ZFhvNENMb2hIcTc4Q201QnpTR0owTVlZT1NaLXFlUWYzUVpqaEttd1g3Yl9DTG85cFExZlYxZmRzODgyNk5VOWcxTFF5cjR2QzE3dGxoQkR1QXJsYkRiVUZaRFN1WnFrTWZZMzFkS1NQdGR2MXlROVdnSHlIZngzaktkQQEB [PROFI] 6/8-13:07:05.288571 20041 EMailProxy [smtps:1] Receive C:235 2.7.0 Accepted [PROFI] 6/8-13:07:05.289737 20041 EMailProxy CAccount::BuildEnhancedBody: Content Type array 'text/html', body:
Thanks.