seems to stall when syncing with gmail

archenemies commented 4 years ago

General informations

system/distribution (with version): Arch Linux - Linux 5.5.2-arch1-1 #1 SMP PREEMPT Tue, 04 Feb 2020 18:56:18 +0000 x86_64 GNU/Linux
offlineimap version (offlineimap -V):

$ offlineimap -V offlineimap v7.3.2, imaplib2 v2.101 (bundled), Python v2.7.17, OpenSSL 1.1.1d 10 Sep 2019
Python version:
server name or domain: gmail.com
CLI options: -d ALL

Configuration file offlineimaprc

Adapted from arch wiki:

[general]
# List of accounts to be synced, separated by a comma.
accounts = gmail

[Account gmail]
# Identifier for the local repository; e.g. the maildir to be synced via IMAP.
localrepository = gmail-local
# Identifier for the remote repository; i.e. the actual IMAP, usually non-local.
remoterepository = gmail-remote

[Repository gmail-local]
# OfflineIMAP supports Maildir, GmailMaildir, and IMAP for local repositories.
type = Maildir
# Where should the mail be placed?
localfolders = ~/mail/offlineimap/gmail/

[Repository gmail-remote]
# Remote repos can be IMAP or Gmail, the latter being a preconfigured IMAP.
#remotehost = gmail.com
type=Gmail
remoteuser=me@gmail.com
remotepass=XXXXX
# nametrans = lambda foldername: re.sub ('^\[gmail\]', 'bak',
#                                re.sub ('sent_mail', 'sent',
#                                re.sub ('starred', 'flagged',
#                                re.sub (' ', '_', foldername.lower()))))
folderfilter = lambda foldername: foldername not in ['[Gmail]/All Mail', 'killed', 'junk']

sslcacertfile = /etc/ssl/certs/ca-certificates.crt
ssl_version = tls1_2

Logs, error

Not sure where these come from

Steps to reproduce the error

Start offlineimap and wait several hours

$ offlineimap -d ALL
...
[imap]: savemessage: date: "28-Feb-2020 07:33:35 -0800", content: 'X-Gmail-Received: 1b4748131d35256839f4fae5e65fba94fb9ac52d
Delivered-To: me@gmail.com
Received: by 10.38.209.45 with SMTP id h45cs5876rng;
  ...s sent to the list will be ignored/filtered.

'
Folder bak/trash [acc: gmail]:
 Syncing messages Gmail[bak/trash] -> Maildir[bak.trash]
 Syncing messages Maildir[bak.trash] -> Gmail[bak/trash]
Folder INBOX [acc: gmail]:
 [thread]: Register new thread 'Folder INBOX [acc: gmail]' (account 'gmail')
Account sync gmail:
 []: Not syncing filtered folder 'junk'[gmail-remote]
 []: Not syncing filtered folder 'killed'[gmail-remote]
[thread]: Unregister thread 'Folder bak/trash [acc: gmail]'
Folder INBOX [acc: gmail]:
 Syncing INBOX: Gmail -> Maildir
 Loading message list for Maildir[INBOX]
 Message list for Maildir[INBOX] loaded: 8975 messages

imap.gmail.com reader:
 [imap]:   43:57.89 imap.gmail.com reader last 20 log messages:

I tried uncommenting the "nametrans" line in the config file, it doesn't seem to help. The last line stays for several hours, and there is no traffic going over the network.

Here is the text which is printed when I press Ctrl-C three times:

oim.txt

archenemies commented 4 years ago

I noticed that when running mbsync I get the following error:

IMAP error: unexpected BYE response: [OVERQUOTA] Account exceeded command or bandwidth limits.

My inbox has 25,000 messages according to mbsync, I'm not sure if that is large or small...

I'm mentioning this in case Offlineimap is getting the same error and just not reporting it back to the user (which I think could be considered a bug).

nicolas33 commented 4 years ago

Interesting. That's the possible root cause, yes.

alexriabtsev commented 3 years ago

hey! I'm getting the same error with mbsync IMAP error: unexpected BYE response: [OVERQUOTA] Account exceeded command or bandwidth limits. Did you manage to find the solution?

archenemies commented 3 years ago

@alexriabtsev I am not sure where the error came from. It went away but I use Lieer and not Offlineimap these days. Gmail is a pain and will periodically disable your sync app so you have to login to the website and re-allow "less secure apps". Sometimes the website UI shows it as enabled but it won't work until you switch it off and on again. I am not sure why it was giving me the overquota message, if it was related to the amount of incoming mail, or the size of my inbox, or what. I wasn't sending anything.

alexriabtsev commented 3 years ago

I found another thread about mbsync option to balance the load on Gmail but looks like this is another argument to stop using it completely.

archenemies commented 3 years ago

@alexriabtsev Yahoo has been fairly stable, I use it with mbsync. Yahoo is less convenient than Gmail if you're using the web interface (the ads are more intrusive), but I haven't had to use it since setting up my automatic mail retrieval. The one thing is that I had to write a script that moves anything in "junk" to "inbox", since Yahoo's spam filtering seems to classify everything from a new sender as spam, and I run that before every fetch. I find that Gmail is still the best option for my main account.

OfflineIMAP / offlineimap