HeaderParseError on mailbox sync

blacklight commented 2 years ago

General informations

system/distribution Tested both on Arch Linux and MacOS:
offlineimap version (offlineimap -V): offlineimap v8.0.0, imaplib2 v3.06, Python v3.9.12, OpenSSL 1.1.1n 15 Mar 2022
Python version: Tested both on 3.9 and 3.10
server name or domain: ProtonMail bridge
CLI options: none

Configuration file offlineimaprc

[general]
accounts = main

[Account main]
localrepository = main-local
remoterepository = main-remote
synclabels = yes

# full refresh, in min
autorefresh = 0.1

# quick refreshs between each full refresh
quick = 10

# update notmuch index after sync
postsynchook = notmuch new

[Repository main-local]
type = Maildir
localfolders = ~/.mail

# delete remote mails that were deleted locally
sync_deletes = yes
ignore-readonly = yes

[Repository main-remote]
type = IMAP
remoteport = 1143
remotehost = 10.0.0.1
remoteuser = *******
remotepass = *******
keepalive = 60
holdconnectionopen = yes
expunge = yes
ssl = no
nametrans = lambda foldername: foldername.split('.')[-1]

[mbnames]
enabled = yes
filename = ~/.mutt/mailboxes
header = "mailboxes "
peritem = "+%(foldername)s"
sep = " "
footer = "\n"

Logs, error

Folder Trash [acc: main]:
 Syncing Trash: IMAP -> Maildir
 Copy message UID -2 (1/2) main-local:Trash -> main-remote:Trash
 ERROR: Copying message -2 [acc: main]
  list index out of range
 ERROR: while syncing Trash [account main]
  list index out of range
 ERROR: ERROR in syncfolder for main folder Trash: Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/_header_value_parser.py", line 2069, in get_msg_id
    token, value = get_dot_atom_text(value)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/_header_value_parser.py", line 1334, in get_dot_atom_text
    raise errors.HeaderParseError("expected atom at a start of "
email.errors.HeaderParseError: expected atom at a start of dot-atom-text but found '>'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/offlineimap-8.0.0-py3.9.egg/offlineimap/accounts.py", line 669, in syncfolder
    localfolder.syncmessagesto(remotefolder, statusfolder)
  File "/usr/local/lib/python3.9/site-packages/offlineimap-8.0.0-py3.9.egg/offlineimap/folder/Base.py", line 1164, in syncmessagesto
    action(dstfolder, statusfolder)
  File "/usr/local/lib/python3.9/site-packages/offlineimap-8.0.0-py3.9.egg/offlineimap/folder/Base.py", line 991, in __syncmessagesto_copy
    self.copymessageto(uid, dstfolder, statusfolder, register=0)
  File "/usr/local/lib/python3.9/site-packages/offlineimap-8.0.0-py3.9.egg/offlineimap/folder/Base.py", line 815, in copymessageto
    new_uid = dstfolder.savemessage(uid, message, flags, rtime)
  File "/usr/local/lib/python3.9/site-packages/offlineimap-8.0.0-py3.9.egg/offlineimap/folder/IMAP.py", line 661, in savemessage
    msg_id = self.getmessageheader(msg, "message-id")
  File "/usr/local/lib/python3.9/site-packages/offlineimap-8.0.0-py3.9.egg/offlineimap/folder/Base.py", line 700, in getmessageheader
    return msg.get(headername)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/message.py", line 471, in get
    return self.policy.header_fetch_parse(k, v)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/policy.py", line 163, in header_fetch_parse
    return self.header_factory(name, value)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/headerregistry.py", line 601, in __call__
    return self[name](name, value)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/headerregistry.py", line 196, in __new__
    cls.parse(value, kwds)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/headerregistry.py", line 529, in parse
    kwds['parse_tree'] = parse_tree = cls.value_parser(value)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/_header_value_parser.py", line 2126, in parse_message_id
    token, value = get_msg_id(value)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/_header_value_parser.py", line 2073, in get_msg_id
    token, value = get_obs_local_part(value)
  File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/_header_value_parser.py", line 1516, in get_obs_local_part
    if (obs_local_part[0].token_type == 'dot' or
IndexError: list index out of range

Steps to reproduce the error

Things were working fine until a few days ago, when the error suddenly started popping out.

I have also tried emptying the Trash folder multiple times from the webapp (most of these errors seem to be caused when syncing the Trash folder), but the issue reproduces later on when I try to move another message to Trash.

blacklight commented 2 years ago

Not sure if it's related, but this is the error I'm regularly getting when I move some messages to Trash:

Folder All Mail [acc: main]:
 ERROR: while syncing All Mail [account main]
  sequence item 0: expected str instance, bytes found
 ERROR: ERROR in syncfolder for main folder All Mail: Traceback (most recent call last):
  File "/usr/local/Cellar/offlineimap/8.0.0_1/libexec/lib/python3.10/site-packages/offlineimap/accounts.py", line 666, in syncfolder
    localfolder.syncmessagesto(remotefolder, statusfolder)
  File "/usr/local/Cellar/offlineimap/8.0.0_1/libexec/lib/python3.10/site-packages/offlineimap/folder/Base.py", line 1164, in syncmessagesto
    action(dstfolder, statusfolder)
  File "/usr/local/Cellar/offlineimap/8.0.0_1/libexec/lib/python3.10/site-packages/offlineimap/folder/Base.py", line 1029, in __syncmessagesto_delete
    dstfolder.deletemessages(deletelist)
  File "/usr/local/Cellar/offlineimap/8.0.0_1/libexec/lib/python3.10/site-packages/offlineimap/folder/IMAP.py", line 1074, in deletemessages
    self.__deletemessages_noconvert(uidlist)
  File "/usr/local/Cellar/offlineimap/8.0.0_1/libexec/lib/python3.10/site-packages/offlineimap/folder/IMAP.py", line 1080, in __deletemessages_noconvert
    self.__addmessagesflags_noconvert(uidlist, set('T'))
  File "/usr/local/Cellar/offlineimap/8.0.0_1/libexec/lib/python3.10/site-packages/offlineimap/folder/IMAP.py", line 987, in __addmessagesflags_noconvert
    self.__processmessagesflags('+', uidlist, flags)
  File "/usr/local/Cellar/offlineimap/8.0.0_1/libexec/lib/python3.10/site-packages/offlineimap/folder/IMAP.py", line 1054, in __processmessagesflags
    self.__processmessagesflags_real(operation,
  File "/usr/local/Cellar/offlineimap/8.0.0_1/libexec/lib/python3.10/site-packages/offlineimap/folder/IMAP.py", line 1019, in __processmessagesflags_real
    'Error with store: %s' % '. '.join(response[1]),
TypeError: sequence item 0: expected str instance, bytes found

noorul commented 1 year ago

I am also getting this error

noorul commented 1 year ago

@BlackLight Did you find any solution for this?

j-dimension commented 1 year ago

I am seeing the same: offlineimap v8.0.0, imaplib2 v3.05, Python v3.10.12, OpenSSL 3.0.2 15 Mar 2022

ERROR: Copying message 56452 [acc: Account1]
  list index out of range
Thread 'Copy message from RemoteIMAP1:INBOX' terminated with exception:
Traceback (most recent call last):
  File "/usr/lib/python3.10/email/_header_value_parser.py", line 2069, in get_msg_id
    token, value = get_dot_atom_text(value)
  File "/usr/lib/python3.10/email/_header_value_parser.py", line 1334, in get_dot_atom_text
    raise errors.HeaderParseError("expected atom at a start of "
email.errors.HeaderParseError: expected atom at a start of dot-atom-text but found

Did anyone have any luck understanding the root cause?

brian-stinar commented 11 months ago

What did you guys come up with for this? We're having the same problems.

j-dimension commented 11 months ago

Nothing so far, except for blacklisting the related message ids.

remipcomaite commented 11 months ago

@BlackLight @brian-stinar @j-dimension I manually applied this patch and it fixed my problem :-) This is a problem in a python library and not in the OfflineIMAP code.

https://github.com/python/cpython/pull/108133

thekix commented 9 months ago

Hi @remipcomaite

thanks for your comment. Could you provide a patch to solve the problem?

Best regards, kix

nickspoons commented 2 months ago

Can confirm that PR #170 resolves this issue for me :tada:

OfflineIMAP / offlineimap3