IMAP issues storing offline with isync.

[mikejzx]

Greetings,

I am running hydroxide on my Linux machine, and using another program called isync (see: 0, 1) to store my e-mail offline in a "Maildir" format on my hard disk.

isync works great with my old Gmail accounts, I haven't run into any issues storing their mail offline. Though when I'm using isync with the hydroxide bridge to store my ProtonMail's e-mail, it stays in sync for about a day or so, then eventually no longer downloads new mail when I attempt to invoke a sync command. I cannot for the life of me work out why this is happening only when using hydroxide.

The strange part is that it all works fine without isync. My e-mail client (NeoMutt) is able to connect to hydroxide's IMAP server great, and all the e-mail is displayed as expected. (The problem with this is obviously that it is still being stored in ProtonMail's servers, whereas I'd like to have it offline.)

I hope someone has some idea of what's going on here. If you'd like me to provide any more information, please ask.

Thank you very much. This program is great. It'd be a shame if it were impossible for me to store my ProtonMail e-mail offline.

[araspik]

I'm having the same issue. isync has worked with GMail and with protonmail-bridge, but in the case of hydroxide it only creates mailboxes, without downloading any e-mail to them. Are there any logs we can provide?

[emersion]

Yeah. Please run hydroxide with -debug to obtain network logs. Make sure to redact the password.

[araspik]

https://paste.sr.ht/~araspik/8a854c56a4b9953068733431a49cef76d078fbe7

[araspik]

That's funny - offlineimap, which is very similar to isync (but is a lot more complicated to set up, and uses Python 2), works perfectly fine. I'm watching it download all my messages right now, and can see message contents in the hydroxide debug log.

[mikejzx]

I noticed a while back that the issue seems to not appear when I configure isync to sync only the Inbox mailbox. (Also works when syncing Sent too. Haven't tried other combinations.)

isync doesn't seem to like it when you try sync mailboxes containing some of the same messages in them in hydroxide's case. (i.e: All Mail, Inbox, etc) Despite working flawlessly with regular mail servers. That's what I've observed at least.

I was considering trying out offlineimap, but I prefer to avoid Python-based programs where possible, for efficiency. Also like @araspik mentioned, it's much more difficult to configure.

[araspik]

I've setup offlineimap for now, but I do agree in avoiding Python programs when possible - are there any other useful alternatives for downloading IMAP->Maildir?

[mikejzx]

I've setup offlineimap for now, but I do agree in avoiding Python programs when possible - are there any other useful alternatives for downloading IMAP->Maildir?

I don't know of any other alternatives. I've been just syncing my Inbox and Sent mailboxes with isync since early this year, and haven't actually run into any problems yet.

This is what I have set up in my .mbsyncrc for ProtonMail Inbox. (Sent is the same, just with different name, etc)

# PM Inbox
Channel pm-inbox
Master :pm-remote:
Slave :pm-local:
Patterns "INBOX"
Create Slave
Expunge Master
SyncState *
Sync PushFlags PushDelete Pull

Basically I've set it up to only pull changes from the remote, and only push flags & deletes since hydroxide doesn't seem to support pushing messages that are added locally. I also had to make sure that my MUA doesn't attempt to copy the message (Fcc) to the Maildir's Inbox. ProtonMail seems to handle the copying on it's own after you send a message to hydroxide's SMTP server. When I do a sync after sending an e-mail, it appears on it's own in the Sent box.

I should also mention that for some odd reason, I have to run mbsync twice for any new messages to come through. Isn't exactly a huge problem since I just have my mail refresh script run it twice, but this obviously does cause it to take twice as long for a proper sync to complete.

[mikejzx]

Just an update I guess. I switched to offlineimap a while back too, and it seems like it is working great. I haven't run into any issues with it when downloading from multiple folders. It seems like hydroxide just doesn't seem to work too well with isync.

[z0noxz]

I have a working setup using isync. This is my config (::{2;<name>} imports data from pass):

IMAPStore       webmail-2-remote
Host            ::{2;host}
Port            ::{2;imap-port}
User            ::{2;username}
PassCmd         "pass mail/::{2;username}/password"
SSLType         None
AuthMechs       LOGIN

MaildirStore    webmail-2-local
Path            ~/mail/webmail-2-local/
Inbox           ~/mail/webmail-2-local/INBOX
Subfolders      Verbatim

Channel webmail-2
Master          :webmail-2-remote:
Slave           :webmail-2-local:
Create          Both
Expunge         Both
Patterns        *
SyncState       *

My only problem is that I get this error/warning that only seems to be for aesthetics: IMAP command 'CLOSE' returned an error: NO [2000] The IDs is required. I guess the method 'CLOSE' isn't implemented in hydroxide?

[TLATER]

I can also get hydroxide to at least work with isync, the crux is AuthMechs LOGIN. Thanks @z0noxz.

[emersion]

LOGIN is a non-standard auth mechanism. Whatever client you're using should use PLAIN instead.

[z0noxz]

The client is mbsync/isync, and the man page says:

AuthMechs type ...
The list of acceptable authentication mechanisms. In addition to the mechanisms listed in the 
SASL registry (link below), the legacy IMAP LOGIN mechanism is known. The wildcard * represents 
all mechanisms that are deemed secure enough for the current SSLType setting. The actually used 
mechanism is the most secure choice from the intersection of this list, the list supplied by the server, 
and the installed SASL modules. (Default: *)

For som reason the PLAIN mechanism doesn't work for me, and gives the error:

IMAP error: selected SASL mechanism(s) not available;
   selected: PLAIN
   available: EXTERNAL
Note: not using LOGIN because connection is not encrypted;
      use 'AuthMechs LOGIN' explicitly to force it.

and doing so works. At least for me. I'm using isync 1.3.3.

[TLATER]

I no longer seem to need AuthMechs LOGIN. I'm not sure what changed, since I'm on version 0.2.15.

My isync configuration:

# Generated by Home Manager.

IMAPAccount tlater.net
Host 127.0.0.1
PassCmd "/nix/store/9mphn4gs656g53734lrn9yyxgjwf826h-read-sops-1.0/bin/read-sops hydroxide"
Port 1143
SSLType None
User tlater

IMAPStore tlater.net-remote
Account tlater.net

MaildirStore tlater.net-local
Inbox /home/tlater/.local/share/mail/tlater.net/Inbox
Path /home/tlater/.local/share/mail/tlater.net/
SubFolders Verbatim

Channel tlater.net
Create Slave
Expunge Both
Master :tlater.net-remote:
Patterns *
Remove Slave
Slave :tlater.net-local:
SyncState *

[stelgenhof]

I am experiencing similar issues using isync and Protonmail. When starting new, my Inbox gets downloaded completely; all mails are there. After some time some mails seem to be missing. Not sure because I've maybe read them using the browsers or mobile app.

Doing another sync, will load the next number of mails, however some others still missing. Not having such issues with Gmail.

Perhaps I should give offlineimap a try, but prefer isync.