Draft errors when composing and sending new email, and then when syncing with offlineimap

[GoogleCodeExporter]

[domain in logs changed to example.com]

What steps will reproduce the problem?
1. From *mu4e-main*, press 'C' to compose a new email. Composition window 
opened, but with an error. From *Messages*:

    Saving file /home/james/Mail/Drafts/cur/20120313-201c19f6830c1ff0f7dd.malbec:2,D...
    Wrote /home/james/Mail/Drafts/cur/20120313-201c19f6830c1ff0f7dd.malbec:2,D
    Toggling flyspell-mode off; better pass an explicit argument.
    Error 11: failed to add path '~/Mail/Drafts/cur/20120313-201c19f6830c1ff0f7dd.malbec:2,D'

2. Compose email and then type C-c C-c to send. Email appears to send 
correctly, but another error seen in *Messages* (after the smtp log), and the 
composition window is left open (I would expect it to close and to return to 
*mu4e-main*).

    Auto-saving...done
    Sending...
    Sending via mail...
    220 smtp.example.com ESMTP Smtpd; Tue, 13 Mar 2012 13:23:47 -0700
    250-smtp.example.com Hello [a.b.c.d], pleased to meet you
    250-ENHANCEDSTATUSCODES
    250-PIPELINING
    250-8BITMIME
    250-SIZE 20971520
    250-DSN
    250-AUTH LOGIN PLAIN
    250-STARTTLS
    250-DELIVERBY
    250 HELP
    235 2.0.0 OK Authenticated
    250 2.1.0 <jladan@example.com>... Sender ok
    250 2.1.5 <cxchen@example.com>... Recipient ok
    354 Enter mail, end with "." on a line by itself
    250 2.0.0 q2DKNl3W021340-q2DKNl3X021340 Message accepted for delivery
    Saving file /home/james/Mail/Drafts/cur/20120313-201c19f6830c1ff0f7dd.malbec:2,D...
    Wrote /home/james/Mail/Drafts/cur/20120313-201c19f6830c1ff0f7dd.malbec:2,D
    if: unknown message ("/home/james/Mail/Drafts/cur/20120313-201c19f6830c1ff0f7dd.malbec:2,D")

3. Run offlineimap to sync email. Error on processing drafts folder:
    ...
     DEBUG[imap]:   27:45.49 mail.example.com reader < KKDK12 NO Message contains invalid header\r\n
    ...
     Thread 'New msg sync from Drafts' terminated with exception:
Traceback (most recent call last):
      File "/usr/lib/pymodules/python2.7/offlineimap/threadutil.py", line 159, in run
        Thread.run(self)
      File "/usr/lib/python2.7/threading.py", line 505, in run
        self.__target(*self.__args, **self.__kwargs)
      File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 248, in syncmessagesto_neguid_msg
        successuid = dstfolder.savemessage(uid, message, flags, rtime)
      File "/usr/lib/pymodules/python2.7/offlineimap/folder/IMAP.py", line 436, in savemessage
        assert(typ == 'OK')
    AssertionError

    Last 23 debug messages logged for New msg sync from Drafts prior to exception:
    imap: savemessage: called
    imap:   27:44.32 New msg sync from Drafts [sync] SELECT ('Drafts',)
    imap:   27:44.32 New msg sync from Drafts state_change_pending.acquire
    imap:   27:44.32 New msg sync from Drafts _request_push(KKDK11, SELECT, {}) = KKDK11
    imap:   27:44.32 New msg sync from Drafts data=KKDK11 SELECT Drafts
    imap:   27:44.32 New msg sync from Drafts SELECT:KKDK11.ready.wait
    imap:   27:45.13 New msg sync from Drafts state => SELECTED
    imap:   27:45.13 New msg sync from Drafts state_change_pending.release
    imap:   27:45.13 New msg sync from Drafts _get_untagged_response(EXISTS) => ['1']
    imap:   27:45.13 New msg sync from Drafts _untagged_response(OK, ?, EXISTS) => ['1']
    imap: savemessage: using date "13-Mar-2012 13:21:35 -0800"
    imap: savemessage: content is: 'From: James Ladan <jladan@example.com>\r\nReply-To: jladan@example.com\r\nTo: \r\nUser-agent: mu4e 0.9.8.1; emacs 23.3.1\r\nSubject: \r\n--text follows this line--\r\n'
    imap:   27:45.13 New msg sync from Drafts [sync] APPEND ('Drafts', '(\\Draft)', '"13-Mar-2012 13:21:35 -0800"')
    imap:   27:45.13 New msg sync from Drafts state_change_pending.acquire
    imap:   27:45.13 New msg sync from Drafts _get_untagged_response(OK) => ['[PERMANENTFLAGS (\\Answered \\Flagged \\Draft \\Deleted \\Seen \\*)]  ']
    imap:   27:45.13 New msg sync from Drafts _request_push(KKDK12, APPEND, {}) = KKDK12
    imap:   27:45.13 New msg sync from Drafts data=KKDK12 APPEND Drafts (\Draft) "13-Mar-2012 13:21:35 -0800" {157}
    imap:   27:45.13 New msg sync from Drafts _request_push(continuation, None, {}) = KKDK13
    imap:   27:45.13 New msg sync from Drafts None:KKDK13.ready.wait
    imap:   27:45.13 New msg sync from Drafts continuation => True, go ahead
    imap:   27:45.13 New msg sync from Drafts write literal size 157
    imap:   27:45.13 New msg sync from Drafts APPEND:KKDK12.ready.wait
    imap:   27:45.49 New msg sync from Drafts state_change_pending.release

4. ~/Mail/Drafts/cur contains both final email and the initial draft before any 
editing with ~ suffix on filename.

What is the expected output? What do you see instead?
- Expect no errors. Saw errors. Expect draft to be removed after sending and 
saved in the configured maildir. Searching with grep in Archive maildir for a 
unique string in the email finds no match (in either 'new' or 'cur').

What version of the product are you using? On what operating system?
- User-agent: mu4e 0.9.8.1; emacs 23.3.1
- Ubuntu 11.10; using Gnome 3.

Configuration snippets:

Emacs:
(when (require 'mu4e nil 'noerror)
  (setq mu4e-user-mail-address-regexp "jladan@example\.com"
    mu4e-maildir       "~/Mail"     ;; top-level Maildir
    mu4e-sent-folder   "/Archive"   ;; where do i keep sent mail?
    mu4e-drafts-folder "/Drafts"    ;; where do i keep half-written mail?
    mu4e-trash-folder  "/Trash"     ;; where do i move deleted mail?
    mu4e-maildir-shortcuts '( ("/INBOX"     . ?i)
                  ("/Archive"   . ?a)
                  ("/lists.tup" . ?t))
    mu4e-html2text-command "html2text"
    mu4e-get-mail-command "pullmail-mu4e"))

"pullmail-mu4e" is a symlink to "pullmail", attached.

Original issue reported on code.google.com by ja...@ladan.ca on 13 Mar 2012 at 9:19

Attachments:

• pullmail

[GoogleCodeExporter]

Forgot to specify mu4e directly in subject line, sorry.

Original comment by ja...@ladan.ca on 13 Mar 2012 at 9:22

[GoogleCodeExporter]

Hi, I think the problem is that your mu4e-maildir is not fully expanded (ie., 
the ~/Mail in your configuration); can you try setting mu4e-maildir to 
"/home/james/Mail" instead? 

(Admittedly, this not very clear from the documentation... next version of mu 
should do this automatically).

Thanks for debugging this!

Original comment by digg...@gmail.com on 14 Mar 2012 at 12:34

[GoogleCodeExporter]

Yep - "/home/james/Mail" does the trick, thanks!

Original comment by ja...@ladan.ca on 16 Mar 2012 at 5:42

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I spoke too soon. The drafts mechanism is fine now, but offlineimap still 
chokes.

The sent email was moved into my Archive folder, but when offlineimap hits it, 
it still complains. (Or maybe the IMAP server doesn't like what it's being 
given.)

-----

Thread 'New msg sync from Archive' terminated with exception:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/offlineimap/threadutil.py", line 159, in run
    Thread.run(self)
  File "/usr/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/Base.py", line 248, in syncmessagesto_neguid_msg
    successuid = dstfolder.savemessage(uid, message, flags, rtime)
  File "/usr/lib/pymodules/python2.7/offlineimap/folder/IMAP.py", line 436, in savemessage
    assert(typ == 'OK')
AssertionError

Last 3 debug messages logged for New msg sync from Archive prior to exception:
imap: savemessage: called
imap: savemessage: using date "16-Mar-2012 10:41:04 -0800"
imap: savemessage: content is: "From: James Ladan <jladan@example.com>\r\nTo: 
James Ladan <james@ladan.ca>\r\nSubject: mu4e test of the day 
(1)\r\nUser-agent: mu4e 0.9.8.1; emacs 23.3.1\r\nReply-To: 
jladan@example.com\r\nDate: Fri, 16 Mar 2012 10:41:03 -0700\r\nMessage-ID: 
<871uos4f68.fsf@example.com>\r\nThis is a test. Let's see what happens when I 
send.\r\n"
Exception in thread mail.example.com reader (most likely raised during 
interpreter shutdown):

Original comment by ja...@ladan.ca on 16 Mar 2012 at 6:01

[GoogleCodeExporter]

I just added a line between "Message-ID" and the body and it worked fine. 
Before the edit, the message body followed the headers without any blank line:

    From: James Ladan <jladan@example.com>
    To: James Ladan <james@ladan.ca>
    Subject: mu4e test of the day (1)
    User-agent: mu4e 0.9.8.1; emacs 23.3.1
    Reply-To: jladan@example.com
    Date: Fri, 16 Mar 2012 10:41:03 -0700
    Message-ID: <871uos4f68.fsf@example.com>
    This is a test. Let's see what happens when I send.

Original comment by ja...@ladan.ca on 16 Mar 2012 at 6:07

[GoogleCodeExporter]

Thanks for checking this! I pushed an update for the newline issue to the git 
repo.

Original comment by digg...@gmail.com on 16 Mar 2012 at 6:48

[GoogleCodeExporter]

Pulled/built/installed. Perfect, thanks. :)

Original comment by ja...@ladan.ca on 16 Mar 2012 at 8:30

[GoogleCodeExporter]

Thanks - closing this.

Original comment by digg...@gmail.com on 20 Mar 2012 at 9:58

• Changed state: Fixed