Handle server side throttling

gbonfiglio commented 4 years ago

General informations

system/distribution (with version): Debian Buster
offlineimap version (offlineimap -V): offlineimap v7.2.3
Python version: v2.7.16
server name or domain: Amazon WorkMail
CLI options: nothing special

Configuration file offlineimaprc

[Repository WorkMail-Remote]
type = IMAP
remotehost = imap.mail.eu-west-1.awsapps.com
remoteuser = REDACTED
readonly = true
maxconnections = 4
ssl = true
sslcacertfile = /etc/ssl/certs/ca-certificates.crt

Logs, error

Thread 'Copy message from WorkMail-Remote:Vault/2007' terminated with exception:
Traceback (most recent call last):
  File "/usr/share/offlineimap/offlineimap/threadutil.py", line 160, in run
    Thread.run(self)
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/share/offlineimap/offlineimap/folder/Base.py", line 839, in copymessageto
    message = self.getmessage(uid)
  File "/usr/share/offlineimap/offlineimap/folder/IMAP.py", line 324, in getmessage
    data = self._fetch_from_imap(str(uid), self.retrycount)
  File "/usr/share/offlineimap/offlineimap/folder/IMAP.py", line 784, in _fetch_from_imap
    imapobj.select(self.getfullIMAPname(), readonly=True)
  File "/usr/share/offlineimap/offlineimap/imaplibutil.py", line 70, in select
    raise OfflineImapError(errstr, severity)
OfflineImapError: Error SELECTing mailbox 'Vault/2007', server reply:
('NO', ['EXAMINE Too many actions per second, please back off (0.210 + 0.000 + 0.209 secs).'])

Steps to reproduce the error

Run a sync with the above server

I've scanned the man back and forth and there doesn't seem to be an option to add a "minimum delay" between commands.

I'm not an expert of IMAP protocol, is this a valid requirement or is the server supposed to slow down / keep operations on hold on its side?

Thanks

gbonfiglio commented 4 years ago

By setting maxconnections = 1 I get less failures (close to none actually), but definitely not zero.

chris001 commented 4 years ago

@gbonfiglio On first look, it seems to be OK for IMAP server to ask for small delay between commands, however I don't see somewhere in the spec that the software should automatically detect the server delay request and then start inserting that delay between each command. Probably this would have to be a global setting of something like 0.5 seconds delay between each command.

gbonfiglio commented 4 years ago

Yeah, that's what I was thinking as in the IMAP protocol there doesn't seem to be a way to "negotiate" delays or slowdown: a global param adding a fixed delay between operations.

Not entirely sure how it will work with multithreading though, and there might be no perfect solution since some servers might throttle on a per-connection basis, some others on a per-account basis (thus causing some possible clashes even with a static delay set).

WebSpider commented 2 years ago

This also occurs when syncing bigger Office365 mailboxes:

OfflineIMAP 7.2.3
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
ERROR: Exceptions occurred during the run!
ERROR: IMAP server 'XXX-Remote' failed to fetch messages UID '202556'. Server responded: NO ['Request is throttled. Suggested Backoff Time: 239883 milliseconds']

Traceback:
  File "/usr/share/offlineimap/offlineimap/folder/Base.py", line 839, in copymessageto
    message = self.getmessage(uid)
  File "/usr/share/offlineimap/offlineimap/folder/IMAP.py", line 324, in getmessage
    data = self._fetch_from_imap(str(uid), self.retrycount)
  File "/usr/share/offlineimap/offlineimap/folder/IMAP.py", line 830, in _fetch_from_imap
    raise OfflineImapError(reason, severity)

OfflineIMAP / offlineimap