Partial reads only leading to invalid multipart parsing

xplodwild commented 6 years ago

Hi there

For some reason, my IMAP emails aren't read entirerly. It seems to read only the first 4096 bytes of the message (which seems to be the peekBufferSize in Go's multipart), which leads to incomplete message bodies, and then an error from multipart: `multipart: expecting a new Part; got line "You can also turn email notifications off:\r\n"

My code is very simple (I'm using go-imap):

r := msg.GetBody("BODY[]")

        if r == nil {
            log.Errorf("Server returned empty body")
            continue
        }

        bodyParser, err := message.Read(r)
        if message.IsUnknownEncoding(err) {
            // Non-fatal error
            log.Noticef("Unknown encoding: %s", err)
        } else if err != nil {
            log.Errorf("Failed to read message: %s", err)
            continue
        }

        if mr := bodyParser.MultipartReader(); mr != nil {
            // This is a multipart message
            for {
                p, err := mr.NextPart()
                if err == io.EOF {
                    break
                } else if err != nil {
                    log.Errorf("Error parsing email part: %s", err) // This throws the error
                    // log.Infof("%s", msgBytes)
                    continue
                }

                t, _, _ := p.Header.ContentType()
                log.Infof("Part is type %s", t)

                part, err := ioutil.ReadAll(p.Body)

                log.Infof("Part: %s", part) // This shows an incomplete email
            }
        } else {
            t, _, _ := bodyParser.Header.ContentType()
            log.Infof("Non-multipart message of type %s", t)
        }

E-mails are simple Slack notification emails. I'm going to see if I can anonymize them and include them in the unit tests.

xplodwild commented 6 years ago

Here's a replacement mailString for mail_test.go that replicates the issue. Sensitive information should be redacted so it's safe to keep.


const mailString = "Return-Path: <bounce+05185b.8e06-JohnDoe=redacted.fr@slack.com>\r\n" +
"Delivered-To: JohnDoe@redacted.fr\r\n" +
"Received: from localhost (localhost [127.0.0.1]) by mail.redacted.fr (Postfix) with ESMTP id AF0CE140EB0 for <JohnDoe@redacted.fr>; Thu,  2 Mar 2017 19:59:37 +0100 (CET)\r\n" +
"Authentication-Results: mail.redacted.fr; dkim=pass reason=\"1024-bit key; unprotected key\" header.d=slack.com header.i=@slack.com header.b=C2uYXsxA; dkim-adsp=pass; dkim-atps=neutral\r\n" +
"X-Virus-Scanned: Debian amavisd-new at mail.redacted.fr\r\n" +
"Received: from mail.redacted.fr ([127.0.0.1]) by localhost (mail.redacted.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TA4aNGHweYO5 for <JohnDoe@redacted.fr>; Thu,  2 Mar 2017 19:59:26 +0100 (CET)\r\n" +
"Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=166.78.69.32; helo=mail-69-32.slack.com; envelope-from=bounce+05185b.8e06-JohnDoe=redacted.fr@slack.com; receiver=JohnDoe@redacted.fr\r\n" +
"Received: from mail-69-32.slack.com (mail-69-32.slack.com [166.78.69.32])\r\n" +
"   (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))\r\n" +
"   (No client certificate requested)\r\n" +
"   by mail.redacted.fr (Postfix) with ESMTPS id 58FC3140EB0\r\n" +
"   for <JohnDoe@redacted.fr>; Thu,  2 Mar 2017 19:59:11 +0100 (CET)\r\n" +
"DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=slack.com; q=dns/txt; s=mailo;\r\n" +
"   t=1488481150; h=Date: Message-Id: Content-Type: MIME-Version: Reply-To:\r\n" +
"From: Subject: To: Sender;\r\n" +
"   bh=BJ7YP6XV6UnZ7/REDACTED+wGWF\r\n" +
"   27XBvI8mgLOtB4W0cVGNu3c0Ur4a3bUfQP79PdGA8OR0yTokDJ04iunE4l/mWUTTabEVTfMW\r\n" +
"   vCDUScWnIQsEmNLZq2Y6izq4IFA=\r\n" +
"DomainKey-Signature: a=rsa-sha1; c=nofws; d=slack.com; s=mailo; q=dns;\r\n" +
"   h=Sender: To: Subject: From: Reply-To: MIME-Version: Content-Type:\r\n" +
"Message-Id: Date;\r\n" +
"   b=JPLyMizhlsjGTVuAxREDACTEDQis1BbWZsm7bvYfmTncdxaiIMJ6CQI6ZcQiu+3\r\n" +
"   tIof9Pd8DvWOr8ed8b4u1zK+cen1Xq1jH/KJlchSPWjiwvirwV1BQ8luo6X3l6Tth+vyfQlQ\r\n" +
"   A4OGMDQF1Ux48hQQPQSICn0FLW1cI=\r\n" +
"Sender: no-reply@slack.com\r\n" +
"X-Mailgun-Sending-Ip: 166.78.69.32\r\n" +
"X-Mailgun-Sid: WyIyMWZjYiIsICJndWlsbGF1bWVAbmV0bG9yLmZyIiwgIjhlMDYiXQ==\r\n" +
"Received: from slack-job-worker838.tinyspeck.com (ec2-54-174-213-95.compute-1.amazonaws.com [54.174.213.95])\r\n" +
"   by mxa.mailgun.org with ESMTP id 58b86b7d.7fb8e05ff660-smtp-out-n02;\r\n" +
"   Thu, 02 Mar 2017 18:59:09 -0000 (UTC)\r\n" +
"Received: by slack-job-worker838.tinyspeck.com (Postfix, from userid 998)\r\n" +
"   id 835E118F0; Thu,  2 Mar 2017 10:59:08 -0800 (PST)\r\n" +
"To: JohnDoe@redacted.fr\r\n" +
"Subject: =?utf-8?Q?=5bSlack=5d_Notifications_from_the_redacted_Team_team_for_Mar_=32nd=2c_=32=30=31=37_at__=37=3a=35=39_PM?=\r\n" +
"From: \"Slack\" <no-reply@slack.com>\r\n" +
"Reply-To: no-reply@slack.com\r\n" +
"MIME-Version: 1.0\r\n" +
"Content-Type: multipart/alternative; boundary=\"__slack_1715508436__\"\r\n" +
"Message-Id: <20170302185908.835E118F0@slack-job-worker838.tinyspeck.com>\r\n" +
"Date: Thu,  2 Mar 2017 10:59:08 -0800 (PST)\r\n" +
"X-redacted-MailScanner-Information: Please contact the ISP for more information\r\n" +
"X-redacted-MailScanner-ID: 58FC3140EB0.A08F1\r\n" +
"X-redacted-MailScanner: Found to be clean\r\n" +
"X-redacted-MailScanner-From: bounce+05185b.8e06-JohnDoe=redacted.fr@slack.com\r\n" +
"\r\n" +
"\r\n" +
"--__slack_1715508436__\r\n" +
"Content-Type: text/plain; charset=\"utf-8\"\r\n" +
"Content-Transfer-Encoding: quoted-printable\r\n" +
"Content-Disposition: inline\r\n" +
"\r\n" +
"Hi JohnDoe,\r\n" +
"\r\n" +
"You have a new direct message from the redacted Team team\r\n" +
"(https://redacted.slack.com/redacted/).\r\n" +
"\r\n" +
"---\r\n" +
"\r\n" +
"@ben\r\n" +
"View in the archives:\r\n" +
"https://redacted.slack.com/redacted/archives/redacted/redacted\r\n" +
"\r\n" +
"CatFacts (7:46 PM, Mar 2nd)\r\n" +
"Cat Fact 20: Cats are often lactose intolerant, so stop givin’ them\r\n" +
"milk!\r\n" +
"\r\n" +
":cat: :cat: :cat:\r\n" +
"\r\n" +
"\r\n" +
"\r\n" +
"* * *\r\n" +
"\r\n" +
"You can snooze these notifications for\r\n" +
"an hour:\r\n" +
"https://redacted.slack.com/redactednotify-mute-1h\r\n" +
"eight hours:\r\n" +
"https://redacted.slack.com/redactednotify-mute-8h\r\n" +
"a day:\r\n" +
"https://redacted.slack.com/redactednotify-mute-1d\r\n" +
"three days:\r\n" +
"https://redacted.slack.com/redactednotify-mute-3d\r\n" +
"or the next week:\r\n" +
"https://redacted.slack.com/redactednotify-mute-7d.\r\n" +
"\r\n" +
"You can also turn email notifications off:\r\n" +
"https://redacted.slack.com/redactednotify.\r\n" +
"\r\n" +
"For more detailed preferences, see your account page:\r\n" +
"https://redacted.slack.com/account.\r\n" +
"\r\n" +
"--__slack_1715508436__\r\n" +
"Content-Type: text/html; charset=\"utf-8\"\r\n" +
"Content-Disposition: inline\r\n" +
"\r\n" +
"<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\r\n" +
"<html xmlns=\"http://www.w3.org/1999/xhtml\"><head><meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\"><meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\"><title>[Slack] Notifications from the redacted Team team for Mar 2nd, 2017 at  7:59 PM</title><style type=\"text/css\">\r\n" +
"\r\n" +
"\r\n" +
"/* Template styling */\r\n" +
"body {\r\n" +
"font-family: \"Helvetica Neue\", Helvetica, Arial, sans-serif;\r\n" +
"width: 100%;\r\n" +
"max-width: 100%;\r\n" +
"font-size: 17px;\r\n" +
"line-height: 24px;\r\n" +
"color: #373737;\r\n" +
"background: #F9F9F9;\r\n" +
"}\r\n" +
"h1, h2, h3, h4 {\r\n" +
"color: #2ab27b;\r\n" +
"margin-bottom: 12px;\r\n" +
"line-height: 26px;\r\n" +
"}\r\n" +
"p, ul, ul li {\r\n" +
"font-size: 17px;\r\n" +
"margin: 0 0 16px;\r\n" +
"line-height: 24px;\r\n" +
"}\r\n" +
"ul {\r\n" +
"margin-bottom: 24px;\r\n" +
"}\r\n" +
"ul li {\r\n" +
"margin-bottom: 8px;\r\n" +
"}\r\n" +
"p.mini {\r\n" +
"font-size: 12px;\r\n" +
"line-height: 18px;\r\n" +
"color: #ABAFB4;\r\n" +
"}\r\n" +
"p.message {\r\n" +
"font-size: 16px;\r\n" +
"line-height: 20px;\r\n" +
"margin-bottom: 4px;\r\n" +
"}\r\n" +
"hr {\r\n" +
"margin: 2rem 0;\r\n" +
"width: 100%;\r\n" +
"border: none;\r\n" +
"border-bottom: 1px solid #ECECEC;\r\n" +
"}\r\n" +
"a, a:link, a:visited, a:active, a:hover {\r\n" +
"font-weight: bold;\r\n" +
"color: #439fe0;\r\n" +
"text-decoration: none;\r\n" +
"word-break: break-word;\r\n" +
"}\r\n" +
"a:active, a:hover {\r\n" +
"text-decoration: underline;\r\n" +
"}\r\n" +
".time {\r\n" +
"font-size: 11px;\r\n" +
"color: #ABAFB4;\r\n" +
"padding-right: 6px;\r\n" +
"}\r\n" +
".emoji {\r\n" +
"vertical-align: bottom;\r\n" +
"}\r\n" +
".avatar {\r\n" +
"border-radius: 2px;\r\n" +
"}\r\n" +
"#footer p {\r\n" +
"margin-top: 16px;\r\n" +
"font-size: 12px;\r\n" +
"}\r\n" +
"\r\n" +
"/* Client-specific Styles */\r\n" +
"#outlook a {padding:0;}\r\n" +
"body{width:100% !important; -webkit-text-size-adjust:100%; -ms-text-size-adjust:100%; margin:0 auto; padding:0;}\r\n" +
".ExternalClass {width:100%;}\r\n" +
".ExternalClass, .ExternalClass p, .ExternalClass span, .ExternalClass font, .ExternalClass td, .ExternalClass div {line-height: 100%;}\r\n" +
"#backgroundTable {margin:0; padding:0; width:100%; line-height: 100% !important;}\r\n" +
"\r\n" +
"/* Some sensible defaults for images\r\n" +
"Bring inline: Yes. */\r\n" +
"img {outline:none; text-decoration:none; -ms-interpolation-mode: bicubic;}\r\n" +
"a img {border:none;}\r\n" +
".image_fix {display:block;}\r\n" +
"\r\n" +
"/* Outlook 07, 10 Padding issue fix\r\n" +
"Bring inline: No.*/\r\n" +
"table td {border-collapse: collapse;}\r\n" +
"\r\n" +
"/* Fix spacing around Outlook 07, 10 tables\r\n" +
"Bring inline: Yes */\r\n" +
"table { border-collapse:collapse; mso-table-lspace:0pt; mso-table-rspace:0pt; }\r\n" +
"\r\n" +
"/* Mobile */\r\n" +
"@media only screen and (max-device-width: 480px) {\r\n" +
"/* Part one of controlling phone number linking for mobile. */\r\n" +
"a[href^=\"tel\"], a[href^=\"sms\"] {\r\n" +
"text-decoration: none;\r\n" +
"color: blue; /* or whatever your want */\r\n" +
"pointer-events: none;\r\n" +
"cursor: default;\r\n" +
"}\r\n" +
"\r\n" +
".mobile_link a[href^=\"tel\"], .mobile_link a[href^=\"sms\"] {\r\n" +
"text-decoration: default;\r\n" +
"color: orange !important;\r\n" +
"pointer-events: auto;\r\n" +
"cursor: default;\r\n" +
"}\r\n" +
"\r\n" +
"}\r\n" +
"\r\n" +
"/* Not all email clients will obey these, but the important ones will */\r\n" +
"@media only screen and (max-width: 480px) {\r\n" +
".card {\r\n" +
"padding: 1rem 0.75rem !important;\r\n" +
"}\r\n" +
".link_option {\r\n" +
"font-size: 14px;\r\n" +
"}\r\n" +
"hr {\r\n" +
"margin: 2rem -0.75rem !important;\r\n" +
"padding-right: 1.5rem !important;\r\n" +
"}\r\n" +
"}\r\n" +
"\r\n" +
"/* More Specific Targeting */\r\n" +
"@media only screen and (min-device-width: 768px) and (max-device-width: 1024px) {\r\n" +
"/* You guessed it, ipad (tablets, smaller screens, etc) */\r\n" +
"/* repeating for the ipad */\r\n" +
"a[href^=\"tel\"], a[href^=\"sms\"] {\r\n" +
"text-decoration: none;\r\n" +
"color: blue; /* or whatever your want */\r\n" +
"pointer-events: none;\r\n" +
"cursor: default;\r\n" +
"}\r\n" +
"\r\n" +
".mobile_link a[href^=\"tel\"], .mobile_link a[href^=\"sms\"] {\r\n" +
"text-decoration: default;\r\n" +
"color: orange !important;\r\n" +
"pointer-events: auto;\r\n" +
"cursor: default;\r\n" +
"}\r\n" +
"}\r\n" +
"\r\n" +
"/* iPhone Retina */\r\n" +
"@media only screen and (-webkit-min-device-pixel-ratio: 2) and (max-device-width: 640px)  {\r\n" +
"/* Must include !important on each style to override inline styles */\r\n" +
"#footer p {\r\n" +
"font-size: 9px;\r\n" +
"}\r\n" +
"}\r\n" +
"\r\n" +
"/* Android targeting */\r\n" +
"@media only screen and (-webkit-device-pixel-ratio:.75){\r\n" +
"/* Put CSS for low density (ldpi) Android layouts in here */\r\n" +
"img {\r\n" +
"max-width: 100%;\r\n" +
"height: auto;\r\n" +
"}\r\n" +
"}\r\n" +
"@media only screen and (-webkit-device-pixel-ratio:1){\r\n" +
"/* Put CSS for medium density (mdpi) Android layouts in here */\r\n" +
"img {\r\n" +
"max-width: 100%;\r\n" +
"height: auto;\r\n" +
"}\r\n" +
"}\r\n" +
"@media only screen and (-webkit-device-pixel-ratio:1.5){\r\n" +
"/* Put CSS for high density (hdpi) Android layouts in here */\r\n" +
"img {\r\n" +
"max-width: 100%;\r\n" +
"height: auto;\r\n" +
"}\r\n" +
"}\r\n" +
"/* Galaxy Nexus */\r\n" +
"@media only screen and (min-device-width : 720px) and (max-device-width : 1280px) {\r\n" +
"img {\r\n" +
"max-width: 100%;\r\n" +
"height: auto;\r\n" +
"}\r\n" +
"body {\r\n" +
"font-size: 16px;\r\n" +
"}\r\n" +
"}\r\n" +
"/* end Android targeting */\r\n" +
"\r\n" +
"\r\n" +
"\r\n" +
"</style></head><body style=\"background: #F9F9F9; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 17px; line-height: 24px; max-width: 100%; width: 100% !important; -ms-text-size-adjust: 100%; -webkit-text-size-adjust: 100%; margin: 0 auto; padding: 0;\">\r\n" +
"<!--[if mso]>\r\n" +
"<style type=\"text/css\">\r\n" +
"\r\n" +
"td { font-family: \"Helvetica Neue\", Helvetica, Arial, sans-serif !important; }\r\n" +
"\r\n" +
"</style>\r\n" +
"<![endif]-->\r\n" +
"\r\n" +
"\r\n" +
"<table width=\"100%\" cellpadding=\"0\" cellspacing=\"0\" border=\"0\"  style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; line-height: 24px; margin: 0; padding: 0; width: 100%; font-size: 17px; color: #373737; background: #F9F9F9;\"><tr><td valign=\"top\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
"<table  width=\"100%\" cellpadding=\"0\" cellspacing=\"0\" border=\"0\" style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt;\"><tr><td valign=\"bottom\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse; padding: 20px 16px 12px;\">\r\n" +
"<div style=\"text-align: center;\">\r\n" +
"<a href=\"https://www.slack.com\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">\r\n" +
"<img src=\"https://redacted.slack.com/redacted/img/email/slack_logo.png\" width=\"120\" height=\"36\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; border: none; margin-left: -1.5rem;\"></a>\r\n" +
"</div>\r\n" +
"</td>\r\n" +
"</tr></table></td>\r\n" +
"</tr><tr><td valign=\"top\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
"\r\n" +
"<table cellpadding=\"32\" cellspacing=\"0\" border=\"0\" align=\"center\"  style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; background: white; border-radius: 0.5rem; margin-bottom: 1rem;\"><tr><td width=\"546\" valign=\"top\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
"<div style=\"max-width: 600px; margin: 0 auto;\">\r\n" +
"<h2 style=\"color: #2ab27b; line-height: 30px; margin-bottom: 12px; margin: 0 0 12px;\">Hi JohnDoe,</h2>\r\n" +
"<p style=\"font-size: 17px; line-height: 24px; margin: 0 0 16px;\">\r\n" +
"\r\n" +
"You have a new direct message from the <strong>redacted Team</strong> team (<a href=\"https://redacted.slack.com/redacted/\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">redacted.slack.com</a>).\r\n" +
"</p>\r\n" +
"\r\n" +
"<hr style=\"border: none; border-bottom: 1px solid #ECECEC; margin: 1.5rem 0; width: 100%;\"><table  width=\"100%\" cellpadding=\"0\" cellspacing=\"0\" border=\"0\" align=\"center\" style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt;\"><h3 style=\"color: #2ab27b; line-height: 18px; margin-bottom: 12px; font-size: 18px; margin: 0 0 8px;\">\r\n" +
"<a href=\"https://redacted.slack.com/redacted/archives/redacted/redacted\" style=\"color: #555549; font-weight: bold; text-decoration: none; word-break: break-word; margin-right: 8px;\">\r\n" +
"@ben\r\n" +
"</a>\r\n" +
"<a href=\"https://redacted.slack.com/redacted/archives/redacted/redacted\" style=\"color: #babbbf; font-weight: normal; text-decoration: none; word-break: break-word; font-size: 12px;\">View in the archives</a>\r\n" +
"</h3>\r\n" +
"<div style=\"margin: 0 0 10px; width: 100%; word-break: break-word; clear: left; font-size: 15px; line-height: 18px; color: #555549; min-height: 36px;\">\r\n" +
"<a href=\"https://redacted.slack.com/redacted/team/CatFacts\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\"><img src=\"https://s3-us-west-2.amazonaws.com/slack-files2/avatars/2015-12-15/16744084547_0ce07b9b66de11dfc9c5_72.png\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; border: none; float: left; margin-right: 8px; border-radius: 4px; display: inline-block; width: 36px; height: 36px;\"></a>\r\n" +
"<div style=\"padding-left: 44px;\">\r\n" +
"<a href=\"https://redacted.slack.com/redacted/team/CatFacts\" style=\"color: #555549; font-weight: bold; text-decoration: none; word-break: break-word; margin-right: 8px;\">CatFacts</a>\r\n" +
"<a href=\"https://redacted.slack.com/redacted/archives/redacted/redacted\" style=\"color: #babbbf; font-weight: normal; text-decoration: none; word-break: break-word; font-size: 12px; white-space: nowrap;\">\r\n" +
"\r\n" +
"7:46 PM, Mar 2nd\r\n" +
"</a><br>\r\n" +
"Cat Fact 20: Cats are often lactose intolerant, so stop givin’ them milk!<br><br><img src=\"https://a.slack-edge.com/d4bf/img/emoji_2015_2/apple/1f431.png\"  title=\":cat:\" width=\"20\" height=\"20\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; vertical-align: bottom;\"><img src=\"https://a.slack-edge.com/d4bf/img/emoji_2015_2/apple/1f431.png\"  title=\":cat:\" width=\"20\" height=\"20\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; vertical-align: bottom;\"><img src=\"https://a.slack-edge.com/d4bf/img/emoji_2015_2/apple/1f431.png\"  title=\":cat:\" width=\"20\" height=\"20\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; vertical-align: bottom;\"></div>\r\n" +
"</div>\r\n" +
"\r\n" +
"\r\n" +
"\r\n" +
"\r\n" +
"</table></div>\r\n" +
"</td>\r\n" +
"</tr></table><table  cellpadding=\"32\" cellspacing=\"0\" border=\"0\" align=\"center\"  style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; background: white; border-radius: 0.5rem; margin-bottom: 1rem;\"><tr><td width=\"546\" valign=\"top\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
"<div style=\"max-width: 600px; margin: 0 auto;\">\r\n" +
"\r\n" +
"<p style=\"font-size: 0.9rem; line-height: 20px; margin: 0 auto 1rem; color: #AAA; text-align: left; max-width: 100%; word-break: break-word; margin-bottom: 0;\">\r\n" +
"\r\n" +
"Snooze these notifications for:\r\n" +
"<a href=\"https://redacted.slack.com/redactednotify-mute-1h\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">an hour</a>,\r\n" +
"<a href=\"https://redacted.slack.com/redactednotify-mute-8h\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">eight hours</a>,\r\n" +
"<a href=\"https://redacted.slack.com/redactednotify-mute-1d\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">a day</a>,\r\n" +
"<a href=\"https://redacted.slack.com/redactednotify-mute-3d\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">three days</a>, or\r\n" +
"<a href=\"https://redacted.slack.com/redactednotify-mute-7d\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">the next week</a>.\r\n" +
"Or, <a href=\"https://redacted.slack.com/redactednotify\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">turn email notifications off</a>. For more detailed preferences, see your <a href=\"https://redacted.slack.com/redacted/account/notifications\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">account page</a>.\r\n" +
"</p>\r\n" +
"\r\n" +
"\r\n" +
"\r\n" +
"</div>\r\n" +
"</td>\r\n" +
"</tr></table></td>\r\n" +
"</tr><tr><td style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
"<table  width=\"100%\" cellpadding=\"0\" cellspacing=\"0\" border=\"0\" align=\"center\" style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; margin-top: 1rem; background: white; color: #989EA6;\"><tr><td style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse; height: 5px; background-image: url('https://a.slack-edge.com/66f9/img/email-ribbon_@2x.png'); background-repeat: repeat-x; background-size: auto 5px;\"></td>\r\n" +
"</tr><tr><td valign=\"top\" align=\"center\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse; padding: 16px 8px 24px;\">\r\n" +
"<div style=\"max-width: 600px; margin: 0 auto;\">\r\n" +
"<p  style=\"font-size: 12px; line-height: 20px; margin: 0 0 16px; margin-top: 16px;\">\r\n" +
"\r\n" +
"Made by <a href=\"https://slack.com\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">Slack Technologies, Inc</a>\r\n" +
" • \r\n" +
"<a href=\"http://slackhq.com\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">\r\n" +
"Our Blog                                       </a><br><a href=\"#\" style=\"color: #989EA6; font-weight: normal; text-decoration: none; word-break: break-word; pointer-events: none;\">\r\n" +
"\r\n" +
"155 5th Street, 6th Floor  •  San Francisco, CA  •  94103\r\n" +
"</a>\r\n" +
"</p>\r\n" +
"</div>\r\n" +
"</td>\r\n" +
"</tr></table></td>\r\n" +
"</tr></table></body></html>\r\n" +
"\r\n" +
"\r\n" +
"--__slack_1715508436__--\r\n"

I'm thinking it's a Golang's mime/multipart bug, but it looks strange as this bug appeared back in Go 1.5 and was fixed in Go 1.6 : https://github.com/golang/go/issues/14085

I tested using Go 1.8 and Go 1.9, both seem to give the same error: multipart: unexpected line in Next(): ".slack.com/account.\r\n"

xplodwild commented 6 years ago

I adapted the mailString above to golang's multipart example (simple reader), each part is read correctly. I'm going to continue investigating the go-message code to find what prevents the multipart from being read properly.

emersion commented 6 years ago

Oh, thanks a lot for your work! I'll trey to debug this tonight too.

xplodwild commented 6 years ago

For reference, here is the code I used to test Golang's multipart:

package main

import (
    "fmt"
    "io"
    "io/ioutil"
    "log"
    "mime"
    "mime/multipart"
    "net/mail"
    "strings"
    "net/textproto"
    "bufio"
)

const mailHeader = "Return-Path: <bounce+05185b.8e06-JohnDoe=redacted.fr@slack.com>\r\n" +
    "Delivered-To: JohnDoe@redacted.fr\r\n" +
    "Received: from localhost (localhost [127.0.0.1]) by mail.redacted.fr (Postfix) with ESMTP id AF0CE140EB0 for <JohnDoe@redacted.fr>; Thu,  2 Mar 2017 19:59:37 +0100 (CET)\r\n" +
    "Authentication-Results: mail.redacted.fr; dkim=pass reason=\"1024-bit key; unprotected key\" header.d=slack.com header.i=@slack.com header.b=C2uYXsxA; dkim-adsp=pass; dkim-atps=neutral\r\n" +
    "X-Virus-Scanned: Debian amavisd-new at mail.redacted.fr\r\n" +
    "Received: from mail.redacted.fr ([127.0.0.1]) by localhost (mail.redacted.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TA4aNGHweYO5 for <JohnDoe@redacted.fr>; Thu,  2 Mar 2017 19:59:26 +0100 (CET)\r\n" +
    "Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=166.78.69.32; helo=mail-69-32.slack.com; envelope-from=bounce+05185b.8e06-JohnDoe=redacted.fr@slack.com; receiver=JohnDoe@redacted.fr\r\n" +
    "Received: from mail-69-32.slack.com (mail-69-32.slack.com [166.78.69.32])\r\n" +
    "   (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))\r\n" +
    "   (No client certificate requested)\r\n" +
    "   by mail.redacted.fr (Postfix) with ESMTPS id 58FC3140EB0\r\n" +
    "   for <JohnDoe@redacted.fr>; Thu,  2 Mar 2017 19:59:11 +0100 (CET)\r\n" +
    "DKIM-Signature: a=rsa-sha256; v=1; c=relaxed/relaxed; d=slack.com; q=dns/txt; s=mailo;\r\n" +
    "   t=1488481150; h=Date: Message-Id: Content-Type: MIME-Version: Reply-To:\r\n" +
    "From: Subject: To: Sender;\r\n" +
    "   bh=BJ7YP6XV6UnZ7/REDACTED+wGWF\r\n" +
    "   27XBvI8mgLOtB4W0cVGNu3c0Ur4a3bUfQP79PdGA8OR0yTokDJ04iunE4l/mWUTTabEVTfMW\r\n" +
    "   vCDUScWnIQsEmNLZq2Y6izq4IFA=\r\n" +
    "DomainKey-Signature: a=rsa-sha1; c=nofws; d=slack.com; s=mailo; q=dns;\r\n" +
    "   h=Sender: To: Subject: From: Reply-To: MIME-Version: Content-Type:\r\n" +
    "Message-Id: Date;\r\n" +
    "   b=JPLyMizhlsjGTVuAxREDACTEDQis1BbWZsm7bvYfmTncdxaiIMJ6CQI6ZcQiu+3\r\n" +
    "   tIof9Pd8DvWOr8ed8b4u1zK+cen1Xq1jH/KJlchSPWjiwvirwV1BQ8luo6X3l6Tth+vyfQlQ\r\n" +
    "   A4OGMDQF1Ux48hQQPQSICn0FLW1cI=\r\n" +
    "Sender: no-reply@slack.com\r\n" +
    "X-Mailgun-Sending-Ip: 166.78.69.32\r\n" +
    "X-Mailgun-Sid: WyIyMWZjYiIsICJndWlsbGF1bWVAbmV0bG9yLmZyIiwgIjhlMDYiXQ==\r\n" +
    "Received: from slack-job-worker838.tinyspeck.com (ec2-54-174-213-95.compute-1.amazonaws.com [54.174.213.95])\r\n" +
    "   by mxa.mailgun.org with ESMTP id 58b86b7d.7fb8e05ff660-smtp-out-n02;\r\n" +
    "   Thu, 02 Mar 2017 18:59:09 -0000 (UTC)\r\n" +
    "Received: by slack-job-worker838.tinyspeck.com (Postfix, from userid 998)\r\n" +
    "   id 835E118F0; Thu,  2 Mar 2017 10:59:08 -0800 (PST)\r\n" +
    "To: JohnDoe@redacted.fr\r\n" +
    "Subject: =?utf-8?Q?=5bSlack=5d_Notifications_from_the_redacted_Team_team_for_Mar_=32nd=2c_=32=30=31=37_at__=37=3a=35=39_PM?=\r\n" +
    "From: \"Slack\" <no-reply@slack.com>\r\n" +
    "Reply-To: no-reply@slack.com\r\n" +
    "MIME-Version: 1.0\r\n" +
    "Content-Type: multipart/alternative; boundary=\"__slack_1715508436__\"\r\n" +
    "Message-Id: <20170302185908.835E118F0@slack-job-worker838.tinyspeck.com>\r\n" +
    "Date: Thu,  2 Mar 2017 10:59:08 -0800 (PST)\r\n" +
    "X-redacted-MailScanner-Information: Please contact the ISP for more information\r\n" +
    "X-redacted-MailScanner-ID: 58FC3140EB0.A08F1\r\n" +
    "X-redacted-MailScanner: Found to be clean\r\n" +
    "X-redacted-MailScanner-From: bounce+05185b.8e06-JohnDoe=redacted.fr@slack.com\r\n" +
    "\r\n" +
    "\r\n"

const mailBody = "--__slack_1715508436__\r\n" +
    "Content-Type: text/plain; charset=\"utf-8\"\r\n" +
    "Content-Transfer-Encoding: quoted-printable\r\n" +
    "Content-Disposition: inline\r\n" +
    "\r\n" +
    "Hi JohnDoe, \r\n" +
    "\r\n" +
    "You have a new direct message from the redacted Team team\r\n" +
    "(https://redacted.slack.com/redacted/).\r\n" +
    "\r\n" +
    "---\r\n" +
    "\r\n" +
    "@ben\r\n" +
    "View in the archives:\r\n" +
    "https://redacted.slack.com/redacted/archives/redacted/redacted\r\n" +
    "\r\n" +
    "CatFacts (7:46 PM, Mar 2nd)\r\n" +
    "Cat Fact 20: Cats are often lactose intolerant, so stop givin' them\r\n" +
    "milk!\r\n" +
    "\r\n" +
    ":cat: :cat: :cat:\r\n" +
    "\r\n" +
    "\r\n" +
    "\r\n" +
    "* * *\r\n" +
    "\r\n" +
    "You can snooze these notifications for\r\n" +
    "an hour:\r\n" +
    "https://redacted.slack.com/redactednotify-mute-1h\r\n" +
    "eight hours:\r\n" +
    "https://redacted.slack.com/redactednotify-mute-8h\r\n" +
    "a day:\r\n" +
    "https://redacted.slack.com/redactednotify-mute-1d\r\n" +
    "three days:\r\n" +
    "https://redacted.slack.com/redactednotify-mute-3d\r\n" +
    "or the next week:\r\n" +
    "https://redacted.slack.com/redactednotify-mute-7d.\r\n" +
    "\r\n" +
    "You can also turn email notifications off:\r\n" +
    "https://redacted.slack.com/redactednotify.\r\n" +
    "\r\n" +
    "For more detailed preferences, see your account page:\r\n" +
    "https://redacted.slack.com/account.\r\n" +
    "\r\n" +
    "--__slack_1715508436__\r\n" +
    "Content-Type: text/html; charset=\"utf-8\"\r\n" +
    "Content-Disposition: inline\r\n" +
    "\r\n" +
    "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\r\n" +
    "<html xmlns=\"http://www.w3.org/1999/xhtml\"><head><meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\"><meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\"><title>[Slack] Notifications from the redacted Team team for Mar 2nd, 2017 at  7:59 PM</title><style type=\"text/css\">\r\n" +
    "\r\n" +
    "\r\n" +
    "/* Template styling */\r\n" +
    "body {\r\n" +
    "font-family: \"Helvetica Neue\", Helvetica, Arial, sans-serif;\r\n" +
    "width: 100%;\r\n" +
    "max-width: 100%;\r\n" +
    "font-size: 17px;\r\n" +
    "line-height: 24px;\r\n" +
    "color: #373737;\r\n" +
    "background: #F9F9F9;\r\n" +
    "}\r\n" +
    "h1, h2, h3, h4 {\r\n" +
    "color: #2ab27b;\r\n" +
    "margin-bottom: 12px;\r\n" +
    "line-height: 26px;\r\n" +
    "}\r\n" +
    "p, ul, ul li {\r\n" +
    "font-size: 17px;\r\n" +
    "margin: 0 0 16px;\r\n" +
    "line-height: 24px;\r\n" +
    "}\r\n" +
    "ul {\r\n" +
    "margin-bottom: 24px;\r\n" +
    "}\r\n" +
    "ul li {\r\n" +
    "margin-bottom: 8px;\r\n" +
    "}\r\n" +
    "p.mini {\r\n" +
    "font-size: 12px;\r\n" +
    "line-height: 18px;\r\n" +
    "color: #ABAFB4;\r\n" +
    "}\r\n" +
    "p.message {\r\n" +
    "font-size: 16px;\r\n" +
    "line-height: 20px;\r\n" +
    "margin-bottom: 4px;\r\n" +
    "}\r\n" +
    "hr {\r\n" +
    "margin: 2rem 0;\r\n" +
    "width: 100%;\r\n" +
    "border: none;\r\n" +
    "border-bottom: 1px solid #ECECEC;\r\n" +
    "}\r\n" +
    "a, a:link, a:visited, a:active, a:hover {\r\n" +
    "font-weight: bold;\r\n" +
    "color: #439fe0;\r\n" +
    "text-decoration: none;\r\n" +
    "word-break: break-word;\r\n" +
    "}\r\n" +
    "a:active, a:hover {\r\n" +
    "text-decoration: underline;\r\n" +
    "}\r\n" +
    ".time {\r\n" +
    "font-size: 11px;\r\n" +
    "color: #ABAFB4;\r\n" +
    "padding-right: 6px;\r\n" +
    "}\r\n" +
    ".emoji {\r\n" +
    "vertical-align: bottom;\r\n" +
    "}\r\n" +
    ".avatar {\r\n" +
    "border-radius: 2px;\r\n" +
    "}\r\n" +
    "#footer p {\r\n" +
    "margin-top: 16px;\r\n" +
    "font-size: 12px;\r\n" +
    "}\r\n" +
    "\r\n" +
    "/* Client-specific Styles */\r\n" +
    "#outlook a {padding:0;}\r\n" +
    "body{width:100% !important; -webkit-text-size-adjust:100%; -ms-text-size-adjust:100%; margin:0 auto; padding:0;}\r\n" +
    ".ExternalClass {width:100%;}\r\n" +
    ".ExternalClass, .ExternalClass p, .ExternalClass span, .ExternalClass font, .ExternalClass td, .ExternalClass div {line-height: 100%;}\r\n" +
    "#backgroundTable {margin:0; padding:0; width:100%; line-height: 100% !important;}\r\n" +
    "\r\n" +
    "/* Some sensible defaults for images\r\n" +
    "Bring inline: Yes. */\r\n" +
    "img {outline:none; text-decoration:none; -ms-interpolation-mode: bicubic;}\r\n" +
    "a img {border:none;}\r\n" +
    ".image_fix {display:block;}\r\n" +
    "\r\n" +
    "/* Outlook 07, 10 Padding issue fix\r\n" +
    "Bring inline: No.*/\r\n" +
    "table td {border-collapse: collapse;}\r\n" +
    "\r\n" +
    "/* Fix spacing around Outlook 07, 10 tables\r\n" +
    "Bring inline: Yes */\r\n" +
    "table { border-collapse:collapse; mso-table-lspace:0pt; mso-table-rspace:0pt; }\r\n" +
    "\r\n" +
    "/* Mobile */\r\n" +
    "@media only screen and (max-device-width: 480px) {\r\n" +
    "/* Part one of controlling phone number linking for mobile. */\r\n" +
    "a[href^=\"tel\"], a[href^=\"sms\"] {\r\n" +
    "text-decoration: none;\r\n" +
    "color: blue; /* or whatever your want */\r\n" +
    "pointer-events: none;\r\n" +
    "cursor: default;\r\n" +
    "}\r\n" +
    "\r\n" +
    ".mobile_link a[href^=\"tel\"], .mobile_link a[href^=\"sms\"] {\r\n" +
    "text-decoration: default;\r\n" +
    "color: orange !important;\r\n" +
    "pointer-events: auto;\r\n" +
    "cursor: default;\r\n" +
    "}\r\n" +
    "\r\n" +
    "}\r\n" +
    "\r\n" +
    "/* Not all email clients will obey these, but the important ones will */\r\n" +
    "@media only screen and (max-width: 480px) {\r\n" +
    ".card {\r\n" +
    "padding: 1rem 0.75rem !important;\r\n" +
    "}\r\n" +
    ".link_option {\r\n" +
    "font-size: 14px;\r\n" +
    "}\r\n" +
    "hr {\r\n" +
    "margin: 2rem -0.75rem !important;\r\n" +
    "padding-right: 1.5rem !important;\r\n" +
    "}\r\n" +
    "}\r\n" +
    "\r\n" +
    "/* More Specific Targeting */\r\n" +
    "@media only screen and (min-device-width: 768px) and (max-device-width: 1024px) {\r\n" +
    "/* You guessed it, ipad (tablets, smaller screens, etc) */\r\n" +
    "/* repeating for the ipad */\r\n" +
    "a[href^=\"tel\"], a[href^=\"sms\"] {\r\n" +
    "text-decoration: none;\r\n" +
    "color: blue; /* or whatever your want */\r\n" +
    "pointer-events: none;\r\n" +
    "cursor: default;\r\n" +
    "}\r\n" +
    "\r\n" +
    ".mobile_link a[href^=\"tel\"], .mobile_link a[href^=\"sms\"] {\r\n" +
    "text-decoration: default;\r\n" +
    "color: orange !important;\r\n" +
    "pointer-events: auto;\r\n" +
    "cursor: default;\r\n" +
    "}\r\n" +
    "}\r\n" +
    "\r\n" +
    "/* iPhone Retina */\r\n" +
    "@media only screen and (-webkit-min-device-pixel-ratio: 2) and (max-device-width: 640px)  {\r\n" +
    "/* Must include !important on each style to override inline styles */\r\n" +
    "#footer p {\r\n" +
    "font-size: 9px;\r\n" +
    "}\r\n" +
    "}\r\n" +
    "\r\n" +
    "/* Android targeting */\r\n" +
    "@media only screen and (-webkit-device-pixel-ratio:.75){\r\n" +
    "/* Put CSS for low density (ldpi) Android layouts in here */\r\n" +
    "img {\r\n" +
    "max-width: 100%;\r\n" +
    "height: auto;\r\n" +
    "}\r\n" +
    "}\r\n" +
    "@media only screen and (-webkit-device-pixel-ratio:1){\r\n" +
    "/* Put CSS for medium density (mdpi) Android layouts in here */\r\n" +
    "img {\r\n" +
    "max-width: 100%;\r\n" +
    "height: auto;\r\n" +
    "}\r\n" +
    "}\r\n" +
    "@media only screen and (-webkit-device-pixel-ratio:1.5){\r\n" +
    "/* Put CSS for high density (hdpi) Android layouts in here */\r\n" +
    "img {\r\n" +
    "max-width: 100%;\r\n" +
    "height: auto;\r\n" +
    "}\r\n" +
    "}\r\n" +
    "/* Galaxy Nexus */\r\n" +
    "@media only screen and (min-device-width : 720px) and (max-device-width : 1280px) {\r\n" +
    "img {\r\n" +
    "max-width: 100%;\r\n" +
    "height: auto;\r\n" +
    "}\r\n" +
    "body {\r\n" +
    "font-size: 16px;\r\n" +
    "}\r\n" +
    "}\r\n" +
    "/* end Android targeting */\r\n" +
    "\r\n" +
    "\r\n" +
    "\r\n" +
    "</style></head><body style=\"background: #F9F9F9; color: #373737; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 17px; line-height: 24px; max-width: 100%; width: 100% !important; -ms-text-size-adjust: 100%; -webkit-text-size-adjust: 100%; margin: 0 auto; padding: 0;\">\r\n" +
    "<!--[if mso]>\r\n" +
    "<style type=\"text/css\">\r\n" +
    "\r\n" +
    "td { font-family: \"Helvetica Neue\", Helvetica, Arial, sans-serif !important; }\r\n" +
    "\r\n" +
    "</style>\r\n" +
    "<![endif]-->\r\n" +
    "\r\n" +
    "\r\n" +
    "<table width=\"100%\" cellpadding=\"0\" cellspacing=\"0\" border=\"0\"  style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; line-height: 24px; margin: 0; padding: 0; width: 100%; font-size: 17px; color: #373737; background: #F9F9F9;\"><tr><td valign=\"top\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
    "<table  width=\"100%\" cellpadding=\"0\" cellspacing=\"0\" border=\"0\" style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt;\"><tr><td valign=\"bottom\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse; padding: 20px 16px 12px;\">\r\n" +
    "<div style=\"text-align: center;\">\r\n" +
    "<a href=\"https://www.slack.com\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">\r\n" +
    "<img src=\"https://redacted.slack.com/redacted/img/email/slack_logo.png\" width=\"120\" height=\"36\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; border: none; margin-left: -1.5rem;\"></a>\r\n" +
    "</div>\r\n" +
    "</td>\r\n" +
    "</tr></table></td>\r\n" +
    "</tr><tr><td valign=\"top\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
    "\r\n" +
    "<table cellpadding=\"32\" cellspacing=\"0\" border=\"0\" align=\"center\"  style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; background: white; border-radius: 0.5rem; margin-bottom: 1rem;\"><tr><td width=\"546\" valign=\"top\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
    "<div style=\"max-width: 600px; margin: 0 auto;\">\r\n" +
    "<h2 style=\"color: #2ab27b; line-height: 30px; margin-bottom: 12px; margin: 0 0 12px;\">Hi JohnDoe,</h2>\r\n" +
    "<p style=\"font-size: 17px; line-height: 24px; margin: 0 0 16px;\">\r\n" +
    "\r\n" +
    "You have a new direct message from the <strong>redacted Team</strong> team (<a href=\"https://redacted.slack.com/redacted/\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">redacted.slack.com</a>).\r\n" +
    "</p>\r\n" +
    "\r\n" +
    "<hr style=\"border: none; border-bottom: 1px solid #ECECEC; margin: 1.5rem 0; width: 100%;\"><table  width=\"100%\" cellpadding=\"0\" cellspacing=\"0\" border=\"0\" align=\"center\" style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt;\"><h3 style=\"color: #2ab27b; line-height: 18px; margin-bottom: 12px; font-size: 18px; margin: 0 0 8px;\">\r\n" +
    "<a href=\"https://redacted.slack.com/redacted/archives/redacted/redacted\" style=\"color: #555549; font-weight: bold; text-decoration: none; word-break: break-word; margin-right: 8px;\">\r\n" +
    "@ben\r\n" +
    "</a>\r\n" +
    "<a href=\"https://redacted.slack.com/redacted/archives/redacted/redacted\" style=\"color: #babbbf; font-weight: normal; text-decoration: none; word-break: break-word; font-size: 12px;\">View in the archives</a>\r\n" +
    "</h3>\r\n" +
    "<div style=\"margin: 0 0 10px; width: 100%; word-break: break-word; clear: left; font-size: 15px; line-height: 18px; color: #555549; min-height: 36px;\">\r\n" +
    "<a href=\"https://redacted.slack.com/redacted/team/CatFacts\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\"><img src=\"https://s3-us-west-2.amazonaws.com/slack-files2/avatars/2015-12-15/16744084547_0ce07b9b66de11dfc9c5_72.png\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; border: none; float: left; margin-right: 8px; border-radius: 4px; display: inline-block; width: 36px; height: 36px;\"></a>\r\n" +
    "<div style=\"padding-left: 44px;\">\r\n" +
    "<a href=\"https://redacted.slack.com/redacted/team/CatFacts\" style=\"color: #555549; font-weight: bold; text-decoration: none; word-break: break-word; margin-right: 8px;\">CatFacts</a>\r\n" +
    "<a href=\"https://redacted.slack.com/redacted/archives/redacted/redacted\" style=\"color: #babbbf; font-weight: normal; text-decoration: none; word-break: break-word; font-size: 12px; white-space: nowrap;\">\r\n" +
    "\r\n" +
    "7:46 PM, Mar 2nd\r\n" +
    "</a><br>\r\n" +
    "Cat Fact 20: Cats are often lactose intolerant, so stop givin' them milk!<br><br><img src=\"https://a.slack-edge.com/d4bf/img/emoji_2015_2/apple/1f431.png\"  title=\":cat:\" width=\"20\" height=\"20\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; vertical-align: bottom;\"><img src=\"https://a.slack-edge.com/d4bf/img/emoji_2015_2/apple/1f431.png\"  title=\":cat:\" width=\"20\" height=\"20\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; vertical-align: bottom;\"><img src=\"https://a.slack-edge.com/d4bf/img/emoji_2015_2/apple/1f431.png\"  title=\":cat:\" width=\"20\" height=\"20\" style=\"-ms-interpolation-mode: bicubic; outline: none; text-decoration: none; vertical-align: bottom;\"></div>\r\n" +
    "</div>\r\n" +
    "\r\n" +
    "\r\n" +
    "\r\n" +
    "\r\n" +
    "</table></div>\r\n" +
    "</td>\r\n" +
    "</tr></table><table  cellpadding=\"32\" cellspacing=\"0\" border=\"0\" align=\"center\"  style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; background: white; border-radius: 0.5rem; margin-bottom: 1rem;\"><tr><td width=\"546\" valign=\"top\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
    "<div style=\"max-width: 600px; margin: 0 auto;\">\r\n" +
    "\r\n" +
    "<p style=\"font-size: 0.9rem; line-height: 20px; margin: 0 auto 1rem; color: #AAA; text-align: left; max-width: 100%; word-break: break-word; margin-bottom: 0;\">\r\n" +
    "\r\n" +
    "Snooze these notifications for:\r\n" +
    "<a href=\"https://redacted.slack.com/redactednotify-mute-1h\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">an hour</a>,\r\n" +
    "<a href=\"https://redacted.slack.com/redactednotify-mute-8h\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">eight hours</a>,\r\n" +
    "<a href=\"https://redacted.slack.com/redactednotify-mute-1d\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">a day</a>,\r\n" +
    "<a href=\"https://redacted.slack.com/redactednotify-mute-3d\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">three days</a>, or\r\n" +
    "<a href=\"https://redacted.slack.com/redactednotify-mute-7d\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">the next week</a>.\r\n" +
    "Or, <a href=\"https://redacted.slack.com/redactednotify\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">turn email notifications off</a>. For more detailed preferences, see your <a href=\"https://redacted.slack.com/redacted/account/notifications\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">account page</a>.\r\n" +
    "</p>\r\n" +
    "\r\n" +
    "\r\n" +
    "\r\n" +
    "</div>\r\n" +
    "</td>\r\n" +
    "</tr></table></td>\r\n" +
    "</tr><tr><td style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse;\">\r\n" +
    "<table  width=\"100%\" cellpadding=\"0\" cellspacing=\"0\" border=\"0\" align=\"center\" style=\"border-collapse: collapse; mso-table-lspace: 0pt; mso-table-rspace: 0pt; margin-top: 1rem; background: white; color: #989EA6;\"><tr><td style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse; height: 5px; background-image: url('https://a.slack-edge.com/66f9/img/email-ribbon_@2x.png'); background-repeat: repeat-x; background-size: auto 5px;\"></td>\r\n" +
    "</tr><tr><td valign=\"top\" align=\"center\" style=\"font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif !important; border-collapse: collapse; padding: 16px 8px 24px;\">\r\n" +
    "<div style=\"max-width: 600px; margin: 0 auto;\">\r\n" +
    "<p  style=\"font-size: 12px; line-height: 20px; margin: 0 0 16px; margin-top: 16px;\">\r\n" +
    "\r\n" +
    "Made by <a href=\"https://slack.com\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">Slack Technologies, Inc</a>\r\n" +
    " • \r\n" +
    "<a href=\"http://slackhq.com\" style=\"color: #439fe0; font-weight: bold; text-decoration: none; word-break: break-word;\">\r\n" +
    "Our Blog                                       </a><br><a href=\"#\" style=\"color: #989EA6; font-weight: normal; text-decoration: none; word-break: break-word; pointer-events: none;\">\r\n" +
    "\r\n" +
    "155 5th Street, 6th Floor   San Francisco, CA   94103\r\n" +
    "</a>\r\n" +
    "</p>\r\n" +
    "</div>\r\n" +
    "</td>\r\n" +
    "</tr></table></td>\r\n" +
    "</tr></table></body></html>\r\n" +
    "\r\n" +
    "\r\n" +
    "--__slack_1715508436__--\r\n"

func main() {
    tp := textproto.NewReader(bufio.NewReader(strings.NewReader(mailHeader + mailBody)))

    mimeHeader, err := tp.ReadMIMEHeader()
    if err != nil {
        panic(err)
    }

    msg := &mail.Message{
        Header: map[string][]string(mimeHeader),
        Body:   strings.NewReader(mailHeader + mailBody),
    }
    mediaType, params, err := mime.ParseMediaType(msg.Header.Get("Content-Type"))
    if err != nil {
        log.Fatal(err)
    }
    if strings.HasPrefix(mediaType, "multipart/") {
        mr := multipart.NewReader(msg.Body, params["boundary"])
        for {
            p, err := mr.NextPart()
            if err == io.EOF {
                return
            }
            if err != nil {
                log.Fatal(err)
            }
            slurp, err := ioutil.ReadAll(p)
            if err != nil {
                log.Fatal(err)
            }
            fmt.Printf("Part %q: %q\n", p.Header.Get("Foo"), slurp)
        }
    }

}

This properly outputs both email parts (text/plain and text/html) one Note that you can pass either the separated header/body or both as header/body, they're still parsed correctly.

xplodwild commented 6 years ago

Got it! There is a special unicode/utf-8 character in the mail body. Search for "givin", you'll get two occurrences of "givin’", most noticeably the ’ character, which is not a regular simple quote. It seems that the mail is properly parsed when you replace that character with a regular quote.

When I wrote my multipart-only test (comment above), I replaced that quote manually, thinking it was an editor fault (the email data otherwise comes from a production IMAP server). When put back, the "special quote" gives the following error from multipart:

quotedprintable: invalid unescaped byte 0xe2 in body

I do NOT get this specific error using go-message though, but since the modified go-message unit test works (first comment) when replacing that character with a regular quote, I highly suspect this is the culprit. well, it doesn't work obviously as the output doesn't match, but it doesn't flat out error

You seem to be parsing charset properly though, so what could cause that issue? I'm going to see if I can mitigate this character encoding issue in my multipart-only code.

xplodwild commented 6 years ago

(Not sure if I should edit my post or spam your inbox, sorry :P)

STOP The special character is not directly the culprit. If you take my multipart-only test code, and remove the "slurp, err := ioutil.ReadAll()" part, you will get the same error:

2017/11/06 15:52:28 Error parsing NextPart: multipart: unexpected line in Next(): "d.slack.com/account.\r\n"

It seems like it is somehow due to the fact the body is not properly/entirely drained from the part, and thus NextPart() starts parsing from the middle of the message. Could it be due to the fact go-message reads multipart differently? Perhaps having a particular bytes read somewhere that could trigger the "invalid unescaped byte" silently somewhere, preventing the message part from being read entirerly, and thus afterwards causing the next part to be read from the middle of the previous part?

xplodwild commented 6 years ago

OK. It is indeed a combination of the two issues:

The body contains an "unescaped byte 0xe2", preventing the body from being read (using ioutil.ReadAll) and thus the Reader hasn't completely read the body
Since the body of the current part is not read properly, it is not reaching the next boundary, thus preventing the parsing of that boundary and causing the parsing error.

Regardless, you will get an error if you never drain the body of the current MultiPart body. This can happen often actually if you want to read only a specific part (e.g. only the attachment) but forget/fail to drain the other parts bodies.

The question now is, why ioutil.ReadAll fails on this character? EDIT: This error actually comes from mime/quotedprintable's Reader, not ioutil directly.

xplodwild commented 6 years ago

I have submitted an issue to golang with a simpler repro case: https://github.com/golang/go/issues/22597

emersion commented 6 years ago

Quoted-printable-encoded text is not expected to contain unescaped UTF-8 codepoints, so it's normal that the quotedprintable package complains. However, there should be a way to be more liberal and don't handle this as a fatal error.

emersion commented 6 years ago

According to the Go issue, Go 1.11 will be less strict about unescaped UTF-8 codepoints in quoted-printable text.

emersion / go-message

Partial reads only leading to invalid multipart parsing #13