search

google / mail-importer

Mail Importer for Gmail will upload the contents of a Thunderbird mail archive to Gmail and do its best to preserve the read state, flagged state, and folders of the messages.
Apache License 2.0
253 stars 91 forks source link

Failure when messageID is missing #5

Open shadowkrusha opened 8 years ago

shadowkrusha commented 8 years ago

I'm trying to import some folders from thunderbird and run into issues where the messageid header is missing from the mail.

This results in a null pointer exception: 

Exception in thread "main" java.lang.NullPointerException
    at     to.lean.tools.gmail.importer.local.thunderbird.ThunderbirdLocalMessage.getMessageId(ThunderbirdLocalMessage.java:58)
    at to.lean.tools.gmail.importer.gmail.Mailbox.mapMessageIds(Mailbox.java:112)
    at to.lean.tools.gmail.importer.gmail.GmailSyncer.sync(GmailSyncer.java:70)
    at to.lean.tools.gmail.importer.Importer.importMail(Importer.java:118)
    at to.lean.tools.gmail.importer.Importer.main(Importer.java:83)

Here is an example of a message with the issues:

From - Fri May 14 18:21:23 2004
X-UIDL: 1084486634.2233.mail-07
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <orders@xxxxx>
Delivered-To: iain@xxxxxxxx
X-Envelope-To: <iain@xxxxxxx>
Received: (qmail 2198 invoked from network); 13 May 2004 22:17:13 -0000
Received: from unknown (HELO dcg.neowing.co.jp) (202.234.167.50)
  by mail.iinet.net.au with SMTP; 13 May 2004 22:17:12 -0000
Received: (qmail 31718 invoked from network); 13 May 2004 22:17:08 -0000
Received: from unknown (HELO dopey) (192.168.10.66)
  by 192.168.10.1 with SMTP; 13 May 2004 22:17:08 -0000
From: orders@xxxxx
To: iain@xxxxxxx
Cc: order@xxxxx
Subject: Order Confirmation #10100255047
MIME-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-Mailer: ZOLAR 6.0i

Dear Iain Riley,

Thank you for placing your order from cdjapan.co.jp.

Thoughts on how to re-generate messageid's? I was thinking about some sort of MD5/BASE64 hash based on date/from/to?

iflan commented 8 years ago

I don't currently have a good solution for this.

There are many ways that a message can be malformed, some of which cause the Gmail API to reject the message. A missing message ID is one, but so is an invalid From: header. I'm sure that there are others, too, that I just haven't run into yet. Because of that, I'm loathe to try to generate message IDs or fix From fields inside Mail Importer itself.

Instead, my plan is to move messages like these to a new mbox format file so that they can be edited by hand and retried. If, however, more people have the missing message ID problem, I could be convinced that automating the fix is worthwhile—especially because Mail Importer depends on messages having unique IDs in order to resume aborted imports.

For now, my suggestion would be to edit that message by hand and add a local message ID. You can consult RFC 5322 to see how they suggest generating message IDs. If you want it to be stable, then using the date and subject is probably a reasonable starting point. It looks like your message, though, doesn't have a Date: header, either, though.

muradgaribzada commented 8 years ago

shadowkrusha, How did you solve this problem?

u-pankaj commented 4 years ago

is it resolved ?

iflan commented 4 years ago

Not really. My suggestion is just to edit the original message. I haven't had the time to really make the mail importer better recently, so all of my aspirations from 2016 remain that.