christianbur / getmail

16 stars 6 forks source link

Container keeps rebooting (... and a little explanation how getmail works) #2

Closed o0donder0o closed 2 years ago

o0donder0o commented 3 years ago

Hi,

Thanks for providing this great solution. I need sync to a bunch of outlook and gmail accounts to my mailcow box and getmail seems the only solution to accomplish this efficiently

Tried to implement your getmail solution. Both through the instructions in the mailcow issue, aswell as the updated version on your own repo. For some reason the container keeps on rebooting and it appears no e-mail message are synced to my mailcow mailbox. I enabled debug for both imap as well as lmtp, but can't find any debug logs.

FYI.. I had to declare the value "imap_sync_folder" in my settings.ini to get getmail starting without any issues. I set this to " inbox". but need to sync other folders like " junk" aswell as i will use spamd and sieve rules to take care of the filtering.

[DEFAULT]
imap_hostname:     MUST_BE_DEFINED_INDIVIDUALLY
imap_port:         993
imap_username:     MUST_BE_DEFINED_INDIVIDUALLY
imap_password:     MUST_BE_DEFINED_INDIVIDUALLY
imap_debug:        True
imap_move_enable:  False
imap_move_folder:  getmail
imap_sync_folder:  inbox
lmtp_hostname:     dovecot-mailcow
lmtp_port:         24
lmtp_recipient:    EMAIL@domain.com
lmtp_debug:        True

[EMAIL_outlook.com]
imap_hostname:     outlook.office365.com
imap_username:     EMAIL@outlook.com
imap_password:     pass

[EMAIL_outlook.de]
imap_hostname:     outlook.office365.com
imap_username:     EMAIL@outlook.de
imap_password:     pass
2021-01-26 22:04:03,111 - MainThread - INFO: use config file: ./settings.ini
2021-01-26 22:04:08,114 - Thread-EMAIL_outlook.com - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.com
2021-01-26 22:04:08,114 - Thread-EMAIL_outlook.de - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.de
2021-01-26 22:04:26,770 - MainThread - INFO: use config file: ./settings.ini
2021-01-26 22:04:31,773 - Thread-EMAIL_outlook.com - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.com
2021-01-26 22:04:31,774 - Thread-EMAIL_outlook.de - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.de
2021-01-26 22:04:59,411 - MainThread - INFO: use config file: ./settings.ini
2021-01-26 22:05:04,414 - Thread-EMAIL_outlook.com - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.com
2021-01-26 22:05:04,415 - Thread-EMAIL_outlook.de - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.de
2021-01-26 22:05:46,893 - MainThread - INFO: use config file: ./settings.ini
2021-01-26 22:05:51,897 - Thread-EMAIL_outlook.com - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.com
2021-01-26 22:05:51,898 - Thread-EMAIL_outlook.de - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.de
2021-01-26 22:06:11,412 - MainThread - INFO: use config file: ./settings.ini
2021-01-26 22:06:16,415 - Thread-EMAIL_outlook.com - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.com
2021-01-26 22:06:16,416 - Thread-EMAIL_outlook.de - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.de
2021-01-26 22:07:07,663 - MainThread - INFO: use config file: ./settings.ini
2021-01-26 22:07:12,666 - Thread-EMAIL_outlook.com - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.com
2021-01-26 22:07:12,667 - Thread-EMAIL_outlook.de - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.de
2021-01-26 22:07:56,549 - MainThread - INFO: use config file: ./settings.ini
2021-01-26 22:08:01,551 - Thread-EMAIL_outlook.com - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.com
2021-01-26 22:08:01,552 - Thread-EMAIL_outlook.de - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.de
2021-01-26 22:08:31,823 - MainThread - INFO: use config file: ./settings.ini
2021-01-26 22:08:36,827 - Thread-EMAIL_outlook.com - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.com
2021-01-26 22:08:36,828 - Thread-EMAIL_outlook.de - INFO: Start Getmail - server: outlook.office365.com:993, username: EMAIL@outlook.de
christianbur commented 3 years ago

Please change "imap_sync_folder: inbox" to "imap_sync_folder: INBOX", this could already solve the problem. "lmtp_recipient:" "lmtp_recipient:" must also be adjusted to the email address of the mailcow mailbox.

I simply created two sync entries (imap_sync_folder inbox and junk) for the same mailbox in the config, runs since over a yearn without problems

[EMAIL_outlook.de] imap_hostname: outlook.office365.com imap_username: EMAIL@outlook.de imap_password: pass

INFO: "imap_sync_folder: INBOX" is default

[EMAIL_outlook.de-junk] imap_hostname: outlook.office365.com imap_username: EMAIL@outlook.de imap_password: pass imap_sync_folder: Junk

christianbur commented 3 years ago

Somehow strange the log from you, actually the script should report errors. Please download the getmail_imap2lmtp.py script again, I re-enabled the line "logging.info("Login - status: %s" % login_status)".

wrong "imap_sync_folder" folder: getmail_1 | 2021-01-27 18:43:34,511 - Thread-junk-TEST - ERROR: ERROR: select failed: Mailbox doesn't exist: xxJunk (0.002 + 0.000 + 0.001 secs).

wrong "imap_password": getmail_1 | 2021-01-27 18:47:25,206 - Thread-junk-TEST - ERROR: ERROR: b'[AUTHENTICATIONFAILED] Authentication failed.'

o0donder0o commented 3 years ago

Thanks for getting back on this and sorry for not replying earlier.

Just tried to provided steps.

  1. Replaced inbox to both INBOX and Inbox.
  2. Added a second entry for each mailbox in settings.ini with additional " imap_sync_folder: Junk" line
  3. Updated getmail_imap2lmtp.py to the last version.

FYI.. "lmtp_recipient:" was set to the receiving mailbox, just overwritten in the settings.ini example for anonymization purposes.

Still experience the same behaviour;

Also tried without any result;

2021-02-08 14:03:25,121 - MainThread - INFO: use config file: ./settings.ini,
2021-02-08 14:03:30,124 - Thread-Outlook.com_mailbox1_Inbox - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox1@outlook.com,
2021-02-08 14:03:30,126 - Thread-Outlook.com_mailbox1_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox1@outlook.com,
2021-02-08 14:03:30,126 - Thread-Outlook.de_mailbox2_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox2@outlook.de,
2021-02-08 14:03:30,126 - Thread-Outlook.de_mailbox2_Inbox - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox2@outlook.de,
2021-02-08 14:03:30,523 - Thread-Outlook.de_mailbox2_Inbox - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:03:30,539 - Thread-Outlook.de_mailbox2_Junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:03:30,566 - Thread-Outlook.com_mailbox1_Junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:03:30,629 - Thread-Outlook.de_mailbox2_Junk - INFO: IMAP fetch mail - inital,
2021-02-08 14:03:30,645 - Thread-Outlook.de_mailbox2_Inbox - INFO: IMAP fetch mail - inital,
2021-02-08 14:03:30,671 - Thread-Outlook.com_mailbox1_Junk - INFO: IMAP fetch mail - inital,
2021-02-08 14:03:30,717 - Thread-Outlook.de_mailbox2_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c,
2021-02-08 14:03:30,765 - Thread-Outlook.com_mailbox1_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c,
2021-02-08 14:03:33,014 - Thread-Outlook.com_mailbox1_Inbox - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:03:33,146 - Thread-Outlook.com_mailbox1_Inbox - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:00,369 - MainThread - INFO: use config file: ./settings.ini,
2021-02-08 14:04:05,372 - Thread-Outlook.com_mailbox1_Inbox - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox1@outlook.com,
2021-02-08 14:04:05,373 - Thread-Outlook.com_mailbox1_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox1@outlook.com,
2021-02-08 14:04:05,373 - Thread-Outlook.de_mailbox2_Inbox - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox2@outlook.de,
2021-02-08 14:04:05,374 - Thread-Outlook.de_mailbox2_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox2@outlook.de,
2021-02-08 14:04:05,803 - Thread-Outlook.de_mailbox2_Junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:04:05,807 - Thread-Outlook.com_mailbox1_Junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:04:05,816 - Thread-Outlook.de_mailbox2_Inbox - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:04:05,835 - Thread-Outlook.com_mailbox1_Inbox - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:04:05,900 - Thread-Outlook.de_mailbox2_Junk - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:05,906 - Thread-Outlook.com_mailbox1_Junk - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:05,945 - Thread-Outlook.de_mailbox2_Inbox - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:05,962 - Thread-Outlook.com_mailbox1_Inbox - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:05,994 - Thread-Outlook.de_mailbox2_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c,
2021-02-08 14:04:06,007 - Thread-Outlook.com_mailbox1_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c,
2021-02-08 14:04:52,973 - MainThread - INFO: use config file: ./settings.ini,
2021-02-08 14:04:57,975 - Thread-Outlook.com_mailbox1_Inbox - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox1@outlook.com,
2021-02-08 14:04:57,975 - Thread-Outlook.com_mailbox1_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox1@outlook.com,
2021-02-08 14:04:57,976 - Thread-Outlook.de_mailbox2_Inbox - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox2@outlook.de,
2021-02-08 14:04:57,976 - Thread-Outlook.de_mailbox2_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox2@outlook.de,
2021-02-08 14:04:58,380 - Thread-Outlook.de_mailbox2_Junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:04:58,383 - Thread-Outlook.de_mailbox2_Inbox - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:04:58,394 - Thread-Outlook.com_mailbox1_Inbox - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:04:58,404 - Thread-Outlook.com_mailbox1_Junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:04:58,476 - Thread-Outlook.de_mailbox2_Junk - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:58,501 - Thread-Outlook.de_mailbox2_Inbox - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:58,502 - Thread-Outlook.com_mailbox1_Junk - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:58,515 - Thread-Outlook.com_mailbox1_Inbox - INFO: IMAP fetch mail - inital,
2021-02-08 14:04:58,564 - Thread-Outlook.de_mailbox2_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c,
2021-02-08 14:04:58,603 - Thread-Outlook.com_mailbox1_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c,
2021-02-08 14:05:45,705 - MainThread - INFO: use config file: ./settings.ini,
2021-02-08 14:05:50,708 - Thread-Outlook.com_mailbox1_Inbox - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox1@outlook.com,
2021-02-08 14:05:50,708 - Thread-Outlook.com_mailbox1_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox1@outlook.com,
2021-02-08 14:05:50,709 - Thread-Outlook.de_mailbox2_Inbox - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox2@outlook.de,
2021-02-08 14:05:50,710 - Thread-Outlook.de_mailbox2_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: Mailbox2@outlook.de,
2021-02-08 14:05:51,113 - Thread-Outlook.de_mailbox2_Inbox - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:05:51,114 - Thread-Outlook.de_mailbox2_Junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:05:51,120 - Thread-Outlook.com_mailbox1_Junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:05:51,131 - Thread-Outlook.com_mailbox1_Inbox - INFO: Login - status: b'LOGIN completed.',
2021-02-08 14:05:51,212 - Thread-Outlook.de_mailbox2_Junk - INFO: IMAP fetch mail - inital,
2021-02-08 14:05:51,219 - Thread-Outlook.com_mailbox1_Junk - INFO: IMAP fetch mail - inital,
2021-02-08 14:05:51,240 - Thread-Outlook.de_mailbox2_Inbox - INFO: IMAP fetch mail - inital,
2021-02-08 14:05:51,259 - Thread-Outlook.com_mailbox1_Inbox - INFO: IMAP fetch mail - inital,
2021-02-08 14:05:51,309 - Thread-Outlook.de_mailbox2_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c,
2021-02-08 14:05:51,311 - Thread-Outlook.com_mailbox1_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c,
christianbur commented 3 years ago

What I noticed right away is that the "Junk" mailboxes work correctly, because the "infinite loop" is started there. Please deactivate the "Inbox" mailboxes and put an email into the junk folder on Outlook.de. Then the email should be retrieved correctly (i hope).

christianbur commented 3 years ago

I have now created an Outlook account for myself, and it works without any problems.

getmail_1  | 2021-02-08 20:26:05,064 - Thread-outlook-de-office365-com - INFO: Start Getmail - server: outlook.office365.com:993, username: christian-xxxxxx@outlook.de
getmail_1  | 2021-02-08 20:26:05,853 - Thread-outlook-de-office365-com - INFO: Login - status: b'LOGIN completed.'
getmail_1  | 2021-02-08 20:26:05,970 - Thread-outlook-de-office365-com - INFO: IMAP fetch mail - inital
getmail_1  | 2021-02-08 20:26:06,256 - Thread-outlook-de-office365-com - INFO: LMTP deliver: start -- LMTP host: dovecot-mailcow:24
getmail_1  | 2021-02-08 20:26:06,341 - Thread-outlook-de-office365-com - INFO: LMTP deliver: new eMail from: [Outlook-Team <no-reply@microsoft.com>], subject: [Willkommen bei Ihrem neuen Outlook.com-Konto]
getmail_1  | 2021-02-08 20:26:06,523 - Thread-outlook-de-office365-com - INFO: IMAP delete: delete email (uid: 5)
getmail_1  | 2021-02-08 20:26:06,630 - Thread-outlook-de-office365-com - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c
[DEFAULT]
imap_hostname:     MUST_BE_DEFINED_INDIVIDUALLY
imap_port:         993
imap_username:     MUST_BE_DEFINED_INDIVIDUALLY
imap_password:     MUST_BE_DEFINED_INDIVIDUALLY
imap_debug:        False
imap_move_enable:  False
imap_move_folder:  getmail
imap_sync_folder:  INBOX
lmtp_hostname:     dovecot-mailcow
lmtp_port:         24
lmtp_recipient:    mail@xxxxxxxxxxxxde
lmtp_debug:        False

[outlook-de-office365-com]
imap_hostname:     outlook.office365.com
imap_username:     christian-xxxx@outlook.de
imap_password:     xxxxxxx
o0donder0o commented 3 years ago

Thanks again for the quick reply and effort you put in helping me out. Very much appreciated!

When I just use the "Junk" sync job on one of the mailboxes it appears to work.

2021-02-08 20:28:41,734 - MainThread - INFO: use config file: ./settings.ini
2021-02-08 20:28:46,737 - Thread-Outlook.de_mailbox2_Junk - INFO: Start Getmail - server: outlook.office365.com:993, username: ********************@outlook.de
2021-02-08 20:28:47,126 - Thread-Outlook.de_mailbox2_Junk - INFO: Login - status: b'LOGIN completed.'
2021-02-08 20:28:47,224 - Thread-Outlook.de_mailbox2_Junk - INFO: IMAP fetch mail - inital
2021-02-08 20:28:47,319 - Thread-Outlook.de_mailbox2_Junk - INFO: Join infinite loop and wait for new mails, cancel with Ctrl-c
2021-02-08 20:29:48,076 - Thread-Outlook.de_mailbox2_Junk - INFO: LMTP deliver: start -- LMTP host: dovecot-mailcow:24
2021-02-08 20:29:48,267 - Thread-Outlook.de_mailbox2_Junk - INFO: LMTP deliver: new eMail from: [[********************],], subject: [[********************],]
2021-02-08 20:29:48,449 - Thread-Outlook.de_mailbox2_Junk - INFO: IMAP delete: delete email (uid: 569)
2021-02-08 20:32:35,442 - Thread-Outlook.de_mailbox2_Junk - INFO: LMTP deliver: start -- LMTP host: dovecot-mailcow:24
2021-02-08 20:32:35,936 - Thread-Outlook.de_mailbox2_Junk - INFO: LMTP deliver: new eMail from: [********************], subject: [[********************],]
2021-02-08 20:32:36,129 - Thread-Outlook.de_mailbox2_Junk - INFO: IMAP delete: delete email (uid: 570)
2021-02-08 20:33:34,299 - Thread-Outlook.de_mailbox2_Junk - INFO: LMTP deliver: start -- LMTP host: dovecot-mailcow:24
2021-02-08 20:33:34,406 - Thread-Outlook.de_mailbox2_Junk - INFO: LMTP deliver: new eMail from: [[********************],], subject: [[********************],]
2021-02-08 20:33:34,557 - Thread-Outlook.de_mailbox2_Junk - INFO: IMAP delete: delete email (uid: 571)

So there must be something wrong with my Inbox sync jobs. Will dive into this a more and update you on my findings.

o0donder0o commented 3 years ago

Seems it works for all mailboxes now, both Inbox and Junk.

Perhaps during my testing something odd happend causing the Inbox folder not to get in to syncing state anymore. I've moved all email from the Inbox to the Junk folder on the problematic mailbox and afterwards Inbox would sync as you can see below. Even the emails moved to the Junk folder synced fine afterwards. For all 3 accounts both Inbox and Junk get to the Infinite loop stage.

Happy that's sorted out now. Really impressed by the instant retrieval of emails.

Have another questions though, hope you might be able to answer.

  1. Now it's working I did notice that the email is removed from the source after it's retrieved. It seems there is no setting to disable this. I would like to play around a bit more before setting things up for the final config. Testing was done on non-important mailboxes btw.

  2. How does getmail keep track of the emails that were synced already? Is this related to the deletion of emails in the source or does it use another approach to keep track aswell?

2021-02-08 21:21:27,980 - MainThread - INFO: use config file: ./settings.ini,
2021-02-08 21:21:32,982 - Thread-mailbox1 - INFO: Start Getmail - server: outlook.office365.com:993, username: ***********@outlook.de,
2021-02-08 21:21:32,983 - Thread-mailbox1-junk - INFO: Start Getmail - server: outlook.office365.com:993, username: ***********@outlook.de,
2021-02-08 21:21:32,983 - Thread-mailbox2 - INFO: Start Getmail - server: outlook.office365.com:993, username: ***********@outlook.de,
2021-02-08 21:21:32,984 - Thread-mailbox2-junk - INFO: Start Getmail - server: outlook.office365.com:993, username: ***********@outlook.de,
2021-02-08 21:21:32,984 - Thread-mailbox3 - INFO: Start Getmail - server: outlook.office365.com:993, username: ***********@outlook.de,
2021-02-08 21:21:32,985 - Thread-mailbox3-junk - INFO: Start Getmail - server: outlook.office365.com:993, username: ***********@outlook.de,
2021-02-08 21:21:33,286 - Thread-mailbox2-junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 21:21:33,311 - Thread-mailbox2 - INFO: Login - status: b'LOGIN completed.',
2021-02-08 21:21:33,356 - Thread-mailbox2-junk - INFO: IMAP fetch mail - inital,
2021-02-08 21:21:33,380 - Thread-mailbox3 - INFO: Login - status: b'LOGIN completed.',
2021-02-08 21:21:33,381 - Thread-mailbox1 - INFO: Login - status: b'LOGIN completed.',
2021-02-08 21:21:33,381 - Thread-mailbox3-junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 21:21:33,387 - Thread-mailbox2 - INFO: IMAP fetch mail - inital,
2021-02-08 21:21:33,407 - Thread-mailbox1-junk - INFO: Login - status: b'LOGIN completed.',
2021-02-08 21:21:33,421 - **Thread-mailbox2-junk - INFO: Join infinite loop** and wait for new mails, cancel with Ctrl-c,
2021-02-08 21:21:33,452 - **Thread-mailbox2 - INFO: Join infinite loop** and wait for new mails, cancel with Ctrl-c,
2021-02-08 21:21:33,469 - Thread-mailbox3-junk - INFO: IMAP fetch mail - inital,
2021-02-08 21:21:33,470 - Thread-mailbox3 - INFO: IMAP fetch mail - inital,
2021-02-08 21:21:33,471 - Thread-mailbox1 - INFO: IMAP fetch mail - inital,
2021-02-08 21:21:33,498 - Thread-mailbox1-junk - INFO: IMAP fetch mail - inital,
2021-02-08 21:21:33,559 - **Thread-mailbox1 - INFO: Join infinite loop** and wait for new mails, cancel with Ctrl-c,
2021-02-08 21:21:33,559 - **Thread-mailbox3-junk - INFO: Join infinite loop** and wait for new mails, cancel with Ctrl-c,
2021-02-08 21:21:33,586 - **Thread-mailbox1-junk - INFO: Join infinite loop** and wait for new mails, cancel with Ctrl-c,
2021-02-08 21:21:33,619 - **Thread-mailbox3 - INFO: Join infinite loop** and wait for new mails, cancel with Ctrl-c,
christianbur commented 3 years ago
  1. Now it's working I did notice that the email is removed from the source after it's retrieved. It seems there is no setting to disable this. I would like to play around a bit more before setting things up for the final config. Testing was done on non-important mailboxes btw.

if you use the following setting, a new folder "getmail" will be created in the source mailbox and the emails will be moved there after the sync, deleting will not happen.

imap_move_enable:  True
imap_move_folder:  getmail

With "imap_move_enable: False" the email will be deleted after the sync in the source mailbox.


  1. How does getmail keep track of the emails that were synced already? Is this related to the deletion of emails in the source or does it use another approach to keep track aswell?

I don't actually sync the folder "imap_sync_folder", but I always empty it completely. So all new emails in the folder still have to be synced :-) The goal is to have no more emails in the source mailbox.

christianbur commented 3 years ago

Info: in every retrieved email a header (X-getmail-retrieved-from-mailbox-user) is added, with this you can filter by sieve (Mailcow: Mail Setup -> Filters -> Add Filt)

require "fileinto";
require "regex";
require "body";

...
if header :contains ["X-getmail-retrieved-from-mailbox-user"] ["xxxtestxxx@gmx.de", "xxxtest2xxxx@gmx.de"]
{
    fileinto "INBOX/Getmail_GMX";
}
elsif header :contains ["X-getmail-retrieved-from-mailbox-user"] ["xxxtestxxx@outlook.de", "xxxtest2xxxx@outlook.de"]
{
    fileinto "INBOX/Getmail_Outlook";
}
else
{
  # The rest goes into INBOX
  # default is "implicit keep", we do it explicitly here
  keep;
}
...
o0donder0o commented 3 years ago

Ones I have completed the mailcow configuration and backup has been confirmed to be working :-), deletion will be fine.

One of the reasons I went for Mailcow was to have all my outlooks and gmails combined into one mailbox without having to set up forwards or imap clients syncing from multiple devices. Tried the builtin imapsync first, but it would not perform that well especially when using loads of mailboxes. If getmail wouldn't be able to run for my setup I would probably try another mailserver solution.

Such a shame mailcow won't implement your solutions as it clearly fills in a requirement and therfor would benefit mailcow. Hope they reconsider.

Thanks for all the help and the sieve filter example @christianbur. Awesome work on this addition to mailcow.

s4069b commented 2 years ago

Hi I have a working MailCow server (latest dockerized version) that I don't want to break. But this sounds like what I've been looking for to work with external imap accounts. What are the 'install instructions'? do I first install get mail. then the yml files in /opt/mailcow-dockerized, edit settings.ini and away we go? thanks

christianbur commented 2 years ago

@s4069b https://github.com/christianbur/docker_projects/blob/master/getmail/INSTALL.TXT

s4069b commented 2 years ago

THANK YOU. That's perfect. Sorry for not looking a little harder - I should have found it.