emersion / go-message

✉️ A streaming Go library for the Internet Message Format and mail messages
MIT License
373 stars 108 forks source link

go-message not parsing header #146

Closed mooijtech closed 2 years ago

mooijtech commented 2 years ago

I have this EML file, which I hope you don't mind me pasting here (it's from DNC leaks):

Received: from dncedge1.dnc.org (192.168.185.10) by DNCHUBCAS1.dnc.org
 (192.168.185.12) with Microsoft SMTP Server (TLS) id 14.3.224.2; Wed, 4 May
 2016 03:13:08 -0400
Received: from server555.appriver.com (8.19.118.102) by dncwebmail.dnc.org
 (192.168.10.221) with Microsoft SMTP Server id 14.3.224.2; Wed, 4 May 2016
 03:13:02 -0400
Received: from [10.87.0.112] (HELO inbound.appriver.com)  by
 server555.appriver.com (CommuniGate Pro SMTP 6.0.4)  with ESMTP id 895195133;
 Wed, 04 May 2016 02:13:09 -0500
X-Note-AR-ScanTimeLocal: 5/4/2016 2:13:07 AM
X-Policy: dnc.org
X-Policy: dnc.org
X-Policy: dnc.org
X-Primary: comers@dnc.org
X-Note: This Email was scanned by AppRiver SecureTide
X-Note: SecureTide Build: 4/25/2016 6:59:12 PM UTC
X-ALLOW: ALLOWED SENDER FOUND
X-ALLOW: ADMIN: @bounce.bluestatedigital.com ALLOWED
X-Virus-Scan: V-
X-Note: Spam Tests Failed: 
X-Country-Path: Switzerland->PRIVATE->United States->
X-Note-Sending-IP: 66.151.182.17
X-Note-Reverse-DNS: mta-sample2.bluestatedigital.com
X-Note-Return-Path: contribution@bounce.bluestatedigital.com
X-Note: User Rule Hits: 
X-Note: Global Rule Hits: G275 G276 G277 G278 G282 G283 G294 G406 
X-Note: Encrypt Rule Hits: 
X-Note: Mail Class: ALLOWEDSENDER
X-Note: Headers Injected
Received: from mta-sample2.bluestatedigital.com ([66.151.182.17] verified)  by
 inbound.appriver.com (CommuniGate Pro SMTP 6.1.7)  with ESMTPS id 136452334;
 Wed, 04 May 2016 02:13:07 -0500
Received: from dnc.queue.bluestatedigital.com
 (openvpnas1.colo.bluestatedigital.com [10.0.180.1])    by
 mta-sample2.bluestatedigital.com (Postfix) with ESMTP id 3r08QB2C0HzTQwN;
    Wed,  4 May 2016 03:13:06 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
    d=bounce.bluestatedigital.com; s=bsdkey; t=1462345986;
    bh=RWqL+tIdb99Cwh+KRfoEQYwuJ/I9FL1TRZ03yJDmad8=;
    h=Date:To:From:Subject:Message-ID:MIME-Version:Content-Type;
    b=o0lIEcY8vb9mw8RpADO9GgLnDc5osCU0B8m8K4EZIhpv/hj+4xv51f/O76aHw0yzn
     QT4hrZ+EdNpfQyirXy7xW6eJ55sX0WYxcJSsYFPqbnuXI8m0P3WdoOfM1BmA3LA66H
     ft8wkNrT5uHnk6xxCkOE0o068gQC/HzcqJFGwXjU=
Received: from phpmailer ([178.198.11.98])  by dnc.queue.bluestatedigital.com
 with HTTP (PHPMailer); Wed, 4 May 2016 03:13:06 -0400
Date: Wed, 4 May 2016 03:13:06 -0400
To: <comers@dnc.org>, <kaplanj@dnc.org>, <rauscherr@dnc.org>
From: Contribution <postmaster@my.democrats.org>
Subject: Contribution: Donate to help Democrats (new abroad) / Janet Weiler / 5.00 USD
Message-ID: <185940ba5e1bca4f3cf6062d22964454@dnc.queue.bluestatedigital.com>
X-Mailer: PHPMailer [version 1.71-blue_mailer]
X-blue_mailer-http_host: my.democrats.org
X-blue_mailer-history_ids: 29768632 29768633 29768634
Content-Type: multipart/alternative;
    boundary="b1_185940ba5e1bca4f3cf6062d22964454"
Return-Path: contribution@bounce.bluestatedigital.com
X-MS-Exchange-Organization-AVStamp-Mailbox: MSFTFF;1;0;0 0 0
X-MS-Exchange-Organization-AuthSource: dncedge1.dnc.org
X-MS-Exchange-Organization-AuthAs: Anonymous
MIME-Version: 1.0

--b1_185940ba5e1bca4f3cf6062d22964454
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
X-WatchGuard-AntiVirus: part scanned. clean action=allow

CONTRIBUTION DATA
-----------------
            Page: Donate to help Democrats (new abroad)
            Type: One-time
    Total Amount: $5.00
      First Name: Janet
       Last Name: Weiler
         Address: 45a ch. d'Eysins
                  Nyon,  1260
                  CH
           Phone: 1223668006
           Email: janet.weiler@bluewin.ch
        Employer: United Nations
Employer Address: 
                  ,  
                  US
      Occupation: retired
    Payment Type: MasterCard
       Account #: 6724
   Transaction #: %%TransactionId%%
       User Id #: %%UserId%%
Contribution Key: IHh5dDBcBMsE8-PXax

COMPLIANCE DATA
------------------------------
Check this box to confirm that the following statements are true and accurate:  I am a United States citizen or a permanent resident alien.  This contribution is not made from the general treasury funds of a corporation, labor organization or national bank. This contribution is not made from the treasury of an entity or person who is a federal contractor. This contribution is not made from the funds of a federally registered foreign agent. I am not a minor under the age of 16. The funds I am donating are not being provided to me by another person or entity for the purpose of making this contribution.: 1

CUSTOM FIELD DATA
------------------------------
(Section header: "Additional Information")
Please enter your US Passport # or Green card # below: 514570517

GENERAL INFORMATION
-------------------
      Submit Date: Wed, 04 May 2016 03:12:48 -0400
           Status: Under Review
Client IP Address: 178.198.11.98
   Client Browser: Mozilla/5.0 (Windows NT 6.0; rv:45.0) Gecko/20100101 Firefox/45.0

--b1_185940ba5e1bca4f3cf6062d22964454
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit
X-WatchGuard-AntiVirus: part scanned. clean action=allow

<meta http-equiv="Content-Type" content="text/html; charset=utf-8"><pre>
CONTRIBUTION DATA
-----------------
            Page: Donate to help Democrats (new abroad)
            Type: One-time
    Total Amount: $5.00
      First Name: Janet
       Last Name: Weiler
         Address: 45a ch. d'Eysins
                  Nyon,  1260
                  CH
           Phone: 1223668006
           Email: janet.weiler@bluewin.ch
        Employer: United Nations
Employer Address: 
                  ,  
                  US
      Occupation: retired
    Payment Type: MasterCard
       Account #: 6724
   Transaction #: 15723993
       User Id #: 
Contribution Key: IHh5dDBcBMsE8-PXax

COMPLIANCE DATA
------------------------------
Check this box to confirm that the following statements are true and accurate:  I am a United States citizen or a permanent resident alien.  This contribution is not made from the general treasury funds of a corporation, labor organization or national bank. This contribution is not made from the treasury of an entity or person who is a federal contractor. This contribution is not made from the funds of a federally registered foreign agent. I am not a minor under the age of 16. The funds I am donating are not being provided to me by another person or entity for the purpose of making this contribution.: 1

CUSTOM FIELD DATA
------------------------------
(Section header: &quot;Additional Information&quot;)
Please enter your US Passport # or Green card # below: 514570517

GENERAL INFORMATION
-------------------
      Submit Date: Wed, 04 May 2016 03:12:48 -0400
           Status: Under Review
Client IP Address: 178.198.11.98
   Client Browser: Mozilla/5.0 (Windows NT 6.0; rv:45.0) Gecko/20100101 Firefox/45.0

</pre>

--b1_185940ba5e1bca4f3cf6062d22964454--

But my code doesn't seem to pick up the starting headers:

func main() {
    inputFile, err := os.Open("/home/bot/data/dncleaks/emails/4.eml")

    // Create a new mail reader
    mr, err := mail.CreateReader(inputFile)

    if err != nil {
        log.Fatal(err)
    }

    // Read each mail's part
    for {
        p, err := mr.NextPart()
        if err == io.EOF {
            break
        } else if err != nil {
            log.Fatal(err)
        }

        switch h := p.Header.(type) {
        case *mail.InlineHeader:
            b, _ := ioutil.ReadAll(p.Body)
            log.Printf("Got text: %v\n", string(b))

            fields := p.Header.(*mail.InlineHeader).Fields()

            for fields.Next() {
                log.Printf("Field: %s\n", fields.Key())
            }
        case *mail.AttachmentHeader:
            filename, _ := h.Filename()
            log.Printf("Got attachment: %v\n", filename)
        }
    }
}

What am I doing wrong?

Kind regards, Marten Mooij

mooijtech commented 2 years ago

It only returns the headers:

Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
X-WatchGuard-AntiVirus: part scanned. clean action=allow
mooijtech commented 2 years ago

Fixed it by

    fields := mr.Header.Fields()

        for fields.Next() {
            log.Printf("Field: %s\n", fields.Key())
        }

Thanks for this awesome library.

I'm working on software (SaaS) which presents and analyzes digital evidence to be presented in court. I'm starting with emails so I've implemented the PST file format specification in Golang: https://github.com/mooijtech/go-pst Now I will add EML support thanks to you!